Signed ints packed wrong
EnTerr opened this issue · 3 comments
I ran into this unpleasantry:
>>> struct.pack('b', -1)
"\127" -- should be '\255'
>>> struct.pack('b', -7)
"y" -- should be '\249'
Fix - comment out these 3 lines:
if val < 0 then
val = val + 2 ^ (n * 8 - 1)
end
Hey!
Actually it was supposed to be +256 in case of signed bytes (instead of +128). I believe everything should be fixed now.
Yes, it should work now with your change in pack
from val = val + 2 ^ (n * 8 - 1)
to val = val + 2 ^ (n * 8)
I also think the entire if val < 0 ...
section is unnecessary because of the properties of modulus. Since 2^m % 2^k = 0 for all k<=m, adding or subtracting 2 ^ (n * 8) won't change the result (all further divisions do not reach that high, you do exactly n times * 2^8 each). I.e. you can chomp the number to bytes w/o turning it to positive and that will give the right results (property of 2's complement)
Anyhow, glad i found your module since i need struct in pure lua, will give workout to the other types and likely suggest "fix" for unpack to return last the end position in the string - to match Ierusalimschy
Thank you for providing me with those links. In fact I have to agree with you. We can safely remove that part and it should work just fine.