This is my second official open source contribution for the development community. At last 7 years working with distributed systems, I always had some problems with socket communication, from the protocol (first byte with message size or a terminator at the end of message) to the structure of code to send/receive data.
I had used a lot of socket communication frameworks of many programming languages, but none of them had a communication protocol implementation, just an abstraction for handle connections, incoming data, disconnections, errors, etc… So I always had problem to check if the received data is completed, if is just a chunk, timeouts, even more when the client/server are in communicating over the internet.
The structure consist in 3 parts:
- 8 bytes protocol header:
Actually, the information are spliced in two parts: the first 4 bytes (LE*) contains the all message data (including 8 bytes), the second 4 bytes (LE) contains the size of the header. NOTE: 4 bytes represents the max integer 2147483647, considering this scenario, the protocol can transport (header + message) in max ~2.1GB
*LE = Little Endian
- Message Header:
Consists in a sequence of bytes for specify a header. The library convert the object to a JSON string format, so is very important understand of this section must be a key/value content.
- Message Body:
Can be any kind of data, a byte array, string, an JSON Object or even an image. You can specify the kind os data using a parameter on the header section.
An easy way to understand the sequential structure of data is:
| 8 bytes protocol header | header | body |
00 00 00 12 00 00 00 05 FF FF FF FF FF 00 00 00 00 00 E1 22 FF 66 00 11 22 33
00 00 00 12 00 00 00 05 = those 8 bytes means: the message contains 18 bytes and 5 bytes are reserved for the message header
FF FF FF FF FF = the respective header content
00 00 00 00 00 = the respective body content
While the buffer not complete the minimum size to complete the message, the lib will “hold” the message until it be concluded.
A the present the following languages have the implementation of protocol:
- NodeJS – https://github.com/irineu/eight-protocol