Special symbol to indicate the start of a binary packet?

[neocool]

What would be a good special symbol to indicate the start of packet in my custom winsock application? I want to be unique with respect to binary data that I am going to transfer (mostly multimedia).

Thanks

 

[mehrara]

Well! If I have understood correctlly 47H symbol may solve your problem as it is the start of frame marker in DVB standard!!!

 

[barny451]

The (only) thing that would make any particular byte value a special symbol is how it is interpreted by the applications that send and receive it. So, use something easy to detect, 0 or ff, and you will have to also make sure that this value *is not* sent by your app in any of the multimedia data.

This can be done either by encoding the data so that it is impossible for the value to occur (e.g. a simple encoding would be to transmit all your binary multimedia data encoded into ascii hex, which cannot contain the unique symbol 0 or ff), or by using an 'escape' character: the escape protocol would be to send the special value (lets say it is 0xff) by sending the escape character (lets say it is 0xfe) followed by (say) 0x7f to stand for the 0xff. At the receiving end this sequence is replaced by just the 0xff. Of course you probably also want to send a value of 0xfe in the multimedia data, and this could be done either by expanding a value of 0xfe to two bytes 0xfe 0x7e, or by sending 0xfe 0xfe (i.e. the escape character twice). Bear in mind that all data sent over the connection has to be protected like this; if there are length fields which could take the value ff these must either be encoded or subject to escape sequence coding.

With either of these schemes you can be sure that when special value occurs it denotes the start of a 'packet'.

The other thing to look out for is overall efficiency: if your special value is a common one in the application data, then with the escape approach the amount of data might be increased significantly (approaching double in the worst case) so you might choose not to use 0 or ff as the special code if those values are particularly common in the application data. Of course the encoding approach will also increase the amount of data transferred, by an amount depending on the encoding scheme selected. In the case of ASCII Hex, one byte would be transmitted in two ascii characters, so the data is always doubled in this case. Of course there are much more efficient encoding schemes. Look at yenc for usenet for a more efficient (and complex) example.

If you want to avoid these issues, and the overhead of possibly failing to correctly detect the start of packet at the first go is not too great, then avoid the special symbol approach completely, add a simple start code (non-unique but infequently-occurring in the data), length, and crc following the data which must all be correct for a packet to be valid.

HTH
barny

 

[neocool]

Thanks for the replies.

... avoid the special symbol approach completely, add a simple start code (non-unique but infequently-occurring in the data), length, and crc...

I was thinking about sending a string like "ZZStart of PacketZZ" or something of that sort. By special code, do you mean any cobination of characters as the one above?

 

[barny451]

Yes, that should work fine. It is very unlikely to occur naturally. NOTE it is not impossible; you may get a very rare occasion when a customer complains that their display was complete garbage.

I remember working on maintenance of a system where the customer complained that after rebooting, sometimes the programmable function keys contained complete garbage. It turned out that the developer had protected the 'non-volatile' memory for the function key storage using an 8-bit CRC. In fact this reduced the probability of detecting corruption to 1 in 256; after startup in the morning, about once a year (in terms of working days) the RAM for each key being corrupt was not detected, and given that there were thirty programmable keys on the keyboard, this resulted in fairly frequent misbehaviour, the result of poor attention to the difference between unlikely and impossible.

HTH
Barny

 

[neocool]

Thanks barny451 and mehrara for the replies.
I might come back soon