iryont/lua-struct

Signed ints packed wrong

EnTerr opened this issue · 3 comments

I ran into this unpleasantry:

>>> struct.pack('b', -1)
"\127"  -- should be '\255'
>>> struct.pack('b', -7)
"y"    -- should be '\249'

Fix - comment out these 3 lines:

      if val < 0 then
        val = val + 2 ^ (n * 8 - 1)
      end

Hey!

Actually it was supposed to be +256 in case of signed bytes (instead of +128). I believe everything should be fixed now.

Yes, it should work now with your change in pack from val = val + 2 ^ (n * 8 - 1) to val = val + 2 ^ (n * 8)

I also think the entire if val < 0 ... section is unnecessary because of the properties of modulus. Since 2^m % 2^k = 0 for all k<=m, adding or subtracting 2 ^ (n * 8) won't change the result (all further divisions do not reach that high, you do exactly n times * 2^8 each). I.e. you can chomp the number to bytes w/o turning it to positive and that will give the right results (property of 2's complement)

Anyhow, glad i found your module since i need struct in pure lua, will give workout to the other types and likely suggest "fix" for unpack to return last the end position in the string - to match Ierusalimschy

Thank you for providing me with those links. In fact I have to agree with you. We can safely remove that part and it should work just fine.