Just what is SYSEX?
Why would you want to know?
These and other burning questions will be answered on this very page!
If you use MIDI-controllable musical equipment you've probably seen this dreaded word, SYSEX (for SYStem EXclusive), lurking at the back of your user's manual. Sometimes it will be accompanied by pages of numbers, which look about as interesting as multiplication tables. But if you want to really use the capabilities of your synths and other MIDI equipment, you'll need to learn to use SYSEX. Because, simply put, the SYSEX data in your instrument controls the sound you produce just as the event data you send it controls the notes it plays.
Event Messages vs SYSEX Messages
There are two major classifications for the bits that run down your MIDI cables. The first is channel or "event" messages. These are short little chunks of data, containing one to three bytes each. These messages have a defined purpose in any synth, they turn notes on and off, set controller values such as volume and pan, select programs, and perform other functions common to all MIDI devices. Not every instrument responds to every possible channel message, but if it does respond, it has (more or less) the same effect as it would on another instrument (with the exception of some controller messages).
The second type of data message is called System Exclusive or SYSEX. These messages can range from a few bytes to tens of thousands of bytes in length. And their content is exclusive to and understood by only one machine, or occasionally one family of machines. These messages have several purposes, but they have one thing in common - they are all different. Other than starting with F0 and ending with F7, there is no real standard of any kind. In fact, some equipment manufacturers seem to go out of their way to be sure that every one of their machines uses different SYSEX messages! A machine may have ten, twenty, or even fifty different SYSEX messages that they understand. And in each of those messages may lie a single piece of information or hundreds of pieces of information.
You probably have a general understanding of channel messages if you use MIDI at all, particularly if you use a computer-based sequencer. A sequencer's job is basically the recording and playing back of these messages. Most sequencer programs can deal with SYSEX messages to some extent, as long as they are not too big and you don't care much about what is in them.
Bits and Bytes
All MIDI data consists of a set of bytes called a message. You probably know that a byte is 8 bits of data. MIDI subdivides each of these bytes in a way that makes in possible for the receiving unit to categorize the data into one of two basic types, status or data. The highest-order bit is used for this purpose. Message bytes that set the highest bit to a one are status bytes (which each message begins with), and other bytes in the message (the data bytes) may never set the highest-order bit to a one. The first byte in a message is always a status byte which is used to identify the type of the message. Message types include note on, note off, controller change and other defined length messages. Another type is SYSEX, which is undefined in length, requiring it to have another special status byte to mark its end.
A SYSEX message always starts with a value of F0 and ends with a value of F7. (F0 and F7 are the hexadecimal representations of 8-bit values.) These beginning and ending status bytes have the high-order bit set, but none of the data bytes inside the message can have the high-order bit set. This means that only 7 bits of each SYSEX data byte can be used, since the high-order bit must be zero. An 8-bit value can represent numbers ranging from 0 to 255. A 7-bit value can only represent numbers from 0 to 127.
Having only 7 bits of data per byte makes communicating data via SYSEX a bit difficult. Particularly for the most important SYSEX messages, which are the type we call "patch data" messages. A Patch Data message is simply one that a synth or other MIDI device uses to define a patch on the device. This is a problem because the computers we call synths (yes, that digital synth is basically a special purpose computer) internally use 8-bit bytes just like practically every modern computer. So, to be able to send this data over a MIDI cable, it has to be formatted in a way that enain(s that the high-order bit is always zero. (Actually, some devices circumvert this limitation by restricting their patch parameter data to values that fit within 0-127.) We'll explain transfer formatting in more detail in a subsequent section, but first let's address an important question.
Why MIDI SYSEX?
Why would you want to send this data over the MIDI cable anyway? There are two main reasons. The first is to be able to save this data on a computer or other device so that you can send it back later. The second is that you may want to be able to edit this data in an environment that is more useful for creating new sounds than the usually very limited interface provided on the machine's front panel. To edit or manipulate the data in any way, you must know every detail about its organization inside the data chunk.
Parameter Update Messages
Parameter update messages are used to tell the synth to change a single value in a patch. We're using the term "patch" here generically, it may describe any type of "program", which includes things commonly called programs, presets, combis, voices, performances, effects setups, multis, and several other names. The following example is a typical parameter update message.
All SYSEX message begin with the SysEx Status Byte (F0) and end with the EOX byte (F7). Most contain the Manufacturers ID (identifying maker, such as E-mu, Korg, Roland) and the Unit ID which identifies the model of the target machine, such as Proteus, M1, JV-1080.
The Device ID identifies the MIDI device connected to the MIDI port. The Device ID identifies one of up to 16 MIDI devices connected to a given MIDI port. In the SYSEX message, Device IDs usually range from 0 to 15, which usually map to IDs 1 to 16 on the instrument. In the case of most Roland devices, Device IDs usually range from 16 to 31, which map to IDs 17 to 32 on the instrument. Having a Device ID lets you connect several devices to a single MIDI port. Each device responds to only those SysEx messages containing its Device ID, and ignores all others.
The Command Code is used to identify the type of message it appears in. In this case, it identifies the message as a Parameter Update message. Command codes vary widely by instrument. These codes sometimes identify a message type, and sometimes are essentially the only information in the message. An example would be Mode Change messages, which look like Parameter Update messages except that there is no other data content.
The values up to this point in a Parameter Update message have a lot in common with all SYSEX messages on a particular machine. Therefore, this portion of the message is often called the header.
The Parameter ID value identifies the parameter to be updated. A parameter is just a single value, such as Volume, Pan, Waveform, etc. Sometimes, this ID has a contextually-based meaning. For example, if the device is in Patch mode, a parameter ID of 22 may specify Volume, in Multi mode it may mean Part 2 Pan.
The Parameter Value field contains the parameter's new value.
Most of the components of a parameter update message preceeding the Parameter ID and value are somewhat standard across instruments. This is not the case with the Parameter ID and value. Parameter IDs may be 1, 2 or more bytes long. Parameter values may be 1 or more bytes long, with widely varying formats for the value. Since a single MIDI data byte can only range from 0 - 127 (7 bits of data), values and IDs that have a range beyond 127 must be broken into 2 (or more) bytes. There are several ways this may be done, and just about every possible method has been used by one instrument or another!
Mode and Control Messages
Mode and Control messages typically look about the same as Parameter Update messages. Sometimes, there is no data in a Mode change message other than the command code. A typical Mode Change message might tell the instrument to change to Patch, Multi, or (whatever) mode. Many instruments have only one mode, and don't need this type of message. There are several types of SYSEX messages that could fall into the Control category. The most common is dump request message. This is a message asking the instrument to send (dump) a patch, a bank of patches, or other data elements. These messages often look like a parameter update message with the command code specifying what kind of data is being requested (patch, voice, combi, etc.) and the parameter value specifying the patch desired.
Patch Data Messages
The biggest and baddest SYSEX message is the "patch data" message. As mentioned previously, we're using the term "patch data" to describe any data element used on a particular machine. These include programs, presets, voices, performances, multis, drums and effects setups, and various other elements implemented on a given synth or other MIDI device.
Patch data messages can range in size from less than 30 to more than 1000 bytes. Each instrument
generally has a set format for the first few bytes of a message,
which is called the message header. The Parameter Update
example message shown previously contains a typical header.
The header is followed by the data. Sometimes the data is followed by a checksum.
The message always ends with the End of Exclusive byte (F7).
The "data" portion of a patch data message is the most interesting and complex part. It contains the definition of the patch, which consists of as many as hundreds of separate parameter values. These values can be divided up into bytes, sets of bytes, or bits within bytes. Actually, really fun data sometimes consists of a couple of bits from one byte, and a couple more from another! Those synth programmers have a tremendous sense of humor, one not often fully appreciated by those of us who are trying to use the data in these messages!
We touched on the concept of transfer formatting earlier. This a how we describe the process of sending what is essentially 8-bit data over a 7-bit link. There are, like everything else in SYSEX, several ways this is done. Before we go into some of them, lets clarify one thing. If your use of SYSEX data is simply to get it from an instrument so that it can be sent back later, you may not care about transfer formatting. You can just grab the data as-is and send it back as-was. You only have to unformat the data if you need to access parameter values inside it.
One of the common ways of dealing with this problem is using 8 MIDI bytes to send 7 data bytes. There are two ways this is done. The most common method, used in many Korg synths, is to save up the high-order bits from each of seven bytes and send them as a single byte which is followed by the seven data bytes. Another way to do this, used in many Alesis machines, is to just "pack" the bits as-is into 8 7-bit bytes. In this approach, the first MIDI byte contains 7 bits of the first data byte, the second MIDI byte contains the leftover bit from the first data byte and 6 bits of the second data byte, and so on.
Another common approach is to nibbleize the data. This means breaking each data byte into two MIDI bytes. The high-order nibble (four bits) is sent first, followed by the low-order nibble. (Of course, sometimes the order of the nibbles is reversed!) This is easy to do but it wastes 3 bits in each MIDI data byte. There are a few other less-commonly used methods, and certainly more will be invented in the future!
ChecksumsA checksum is just a special value that is used to detect data transmission errors. In its simplest form, it is just the sum of the data portion of the message, formatted or truncated to a 7-bit value, and then sent at the end of the data (just before the F7). The idea here is that if any data bits were corrupted in transmission, the sum of the bytes will not match the sum that was calculated and placed into the message by the sender. Just like everything else in the world of SYSEX, there are several different ways to calculate checksum values.
That's All For NowWe hope that this has helped you to understand MIDI SYSEX a little better. Terzoid Software is dedicated to making the best SYSEX tools available for the Windows platform.
Last Updated: August 31, 2006 by firstname.lastname@example.org
Copyright © 1995-2014 by Terzoid Software