+ Post New Thread
Results 1 to 11 of 11
  1. #1
    Newbie level 4
    Points: 2,263, Level: 11

    Join Date
    May 2006
    Posts
    7
    Helped
    0 / 0
    Points
    2,263
    Level
    11

    Small Data Packet Compression

    Hi,
    I am reading some data on microcontroller which i want to send out to pc using usb. I need to compress the data so that i can send as much data as possible on usb. One data packet size is 16 bytes which I need to compress to 12 bytes as max. Does anybody know any algorithm which compresses this small packet of data.
    Any help will be appreciated.
    Regards.

  2. #2
    Advanced Member level 5
    Points: 21,803, Level: 35
    barry's Avatar
    Join Date
    Mar 2005
    Location
    California, USA
    Posts
    4,184
    Helped
    921 / 921
    Points
    21,803
    Level
    35

    Re: Small Data Packet Compression

    Do you need lossless compression?



    •   AltAdvertisment

        
       

  3. #3
    Super Moderator
    Points: 65,915, Level: 62
    Achievements:
    7 years registered
    Awards:
    Most Frequent Poster 3rd Helpful Member

    Join Date
    Apr 2014
    Posts
    13,408
    Helped
    3097 / 3097
    Points
    65,915
    Level
    62

    Re: Small Data Packet Compression

    Hi,

    What data format is it now?
    And what information is in the data?

    Klaus
    Please donīt contact me via PM, because there is no time to respond to them. No friend requests. Thank you.



  4. #4
    Newbie level 4
    Points: 2,263, Level: 11

    Join Date
    May 2006
    Posts
    7
    Helped
    0 / 0
    Points
    2,263
    Level
    11

    Re: Small Data Packet Compression

    Hi, Thanks for the reply.
    Yes, it needs to be lossless compression. Also, the data is byte array.



    •   AltAdvertisment

        
       

  5. #5
    Super Moderator
    Points: 65,915, Level: 62
    Achievements:
    7 years registered
    Awards:
    Most Frequent Poster 3rd Helpful Member

    Join Date
    Apr 2014
    Posts
    13,408
    Helped
    3097 / 3097
    Points
    65,915
    Level
    62

    Re: Small Data Packet Compression

    Hi,

    Also, the data is byte array.
    What else can it be?

    Klaus
    Please donīt contact me via PM, because there is no time to respond to them. No friend requests. Thank you.



    •   AltAdvertisment

        
       

  6. #6
    Super Moderator
    Points: 248,101, Level: 100
    Awards:
    1st Helpful Member

    Join Date
    Jan 2008
    Location
    Bochum, Germany
    Posts
    43,150
    Helped
    13128 / 13128
    Points
    248,101
    Level
    100

    Re: Small Data Packet Compression

    Do you understand how data compression works? It's only feasible if the data involves redundancy that can be reduced by recoding, e.g. repeated bytes or more generally repeated patterns. Achieving a guaranteed reduction of small data entities is rather unlikely. What's the nature of your data, beyond being byte arrays?



  7. #7
    Advanced Member level 1
    Points: 3,749, Level: 14

    Join Date
    Feb 2014
    Posts
    478
    Helped
    174 / 174
    Points
    3,749
    Level
    14

    Re: Small Data Packet Compression

    Yeah my thought is that you're probably better off compressing it yourself.

    You have information and knowledge about the data which no general purpose algorithm has access too. Thus you're probably in a better position to cut out extra data than any general purpose algorithm.

    For example if you're transmitting ASCII characters in standard 8-bits but you know you're not using most symbols you might be able to encode them with 7, 6 or even 5 bits. This is knowledge only you have and this is the thought process you need to use to shrink the data format.



  8. #8
    Advanced Member level 5
    Points: 21,803, Level: 35
    barry's Avatar
    Join Date
    Mar 2005
    Location
    California, USA
    Posts
    4,184
    Helped
    921 / 921
    Points
    21,803
    Level
    35

    Re: Small Data Packet Compression

    It's kind of a rule that no lossless compression algorithm will actually compress all files. Asking for a 25% compression for only 16 bytes might be unobtainable.

    For example, do you think you can compress 2 bytes down to one byte?



  9. #9
    Advanced Member level 1
    Points: 3,749, Level: 14

    Join Date
    Feb 2014
    Posts
    478
    Helped
    174 / 174
    Points
    3,749
    Level
    14

    Re: Small Data Packet Compression

    It might be a peace of cake if it's an ascii protocol that could be converted to binary for example.

    Or not...I assume information theory has definitions for this stuff but you can't actually compress information. You can optimize how information is packed into a given container. You can identify and throw out repetition. You can have sender and receiver agree on certain things so they don't need to be retransmitted. Lossy compression can intelligently throw out the least important things. But information is information. If you have X bits of actual information then you need to send X bits. No less.



    •   AltAdvertisment

        
       

  10. #10
    Advanced Member level 5
    Points: 21,803, Level: 35
    barry's Avatar
    Join Date
    Mar 2005
    Location
    California, USA
    Posts
    4,184
    Helped
    921 / 921
    Points
    21,803
    Level
    35

    Re: Small Data Packet Compression

    Quote Originally Posted by asdf44 View Post
    If you have X bits of actual information then you need to send X bits. No less.
    You have managed to totally miss the point of data compression while also contradicting yourself.



  11. #11
    Advanced Member level 1
    Points: 3,749, Level: 14

    Join Date
    Feb 2014
    Posts
    478
    Helped
    174 / 174
    Points
    3,749
    Level
    14

    Re: Small Data Packet Compression

    Quote Originally Posted by barry View Post
    You have managed to totally miss the point of data compression while also contradicting yourself.
    Like I said, there are probably actual terms to use to describe this more precisely....however the point stands. If the OP really needs to transmit 2^(8*16) possibilities then no compression algorithm is going to help.

    Lossless compression algorithms can't throw out actual information: they can only identify bits that are carrying redundant or unnecessary information and throw away those.

    And again: a general purpose algorithm with no special knowledge about the data being transmitted is going to have almost no chance of working on such a small word. Hence the OP is better off compacting the data format for themself.



--[[ ]]--