Eight-Protocol

Hi folks!

This is my second official open source contribution for the development community. At last 7 years working with distributed systems, I always had some problems with socket communication, from the protocol (first byte with message size or a terminator at the end of message) to the structure of code to send/receive data.

I had used a lot of socket communication frameworks of many programming languages, but none of them had a communication protocol implementation, just an abstraction for handle connections, incoming data, disconnections, errors, etc… So I always had problem to check if the received data is completed, if is just a chunk, timeouts,  even more when the client/server are in communicating over the internet.

Protocol specification

The structure consist in 3 parts:

  1. 8 bytes protocol header:
    Actually, the information are spliced in two parts: the first 4 bytes (LE*) contains the all message data (including 8 bytes), the second 4 bytes (LE) contains the size of the header. NOTE: 4 bytes represents the max integer 2147483647, considering this scenario, the protocol can transport (header + message) in max ~2.1GB
    *LE = Little Endian
  2. Message Header:
    Consists in a sequence of bytes for specify a header. The library convert the object to a JSON string format, so is very important understand of this section must be a key/value content.
  3. Message Body:
    Can be any kind of data, a byte array, string, an JSON Object or even an image. You can specify the kind os data using a parameter on the header section.

An easy way to understand the sequential structure of data is:

| 8 bytes protocol header | header | body |

Example:

00 00 00 12 00 00 00 05 FF FF FF FF FF 00 00 00 00 00 E1 22 FF 66 00 11 22 33

00 00 00 12 00 00 00 05 = those 8 bytes means: the message contains 18 bytes and 5 bytes are reserved for the message header

FF FF FF FF FF = the respective header content

00 00 00 00 00 = the respective body content

While the buffer not complete the minimum size to complete the message, the lib will “hold” the message until it be concluded.

Implementations

A the present the following languages have the implementation of protocol:

  • NodeJS – https://github.com/irineu/eight-protocol