Binary.stringToBytes produces incorrect results

[GoogleCodeExporter]

While using crypto-js we have experienced some problems in converting a byte 
string into a list of integers. We occasionally get values bigger than 0xff. 
Which obviously should not happen when converting a bytestring into integers.

We were able to fix the problem by doing a bitwise AND against 0xff before 
outputting the value (see code below). This seemed to fix the issue for us. We 
feel this might be a bug in the library. In case we got it wrong, we'd like to 
gain better understanding about what is happening.

Crypto.charenc.Binary.stringToBytes = function (str) {
  for (var bytes = [], i = 0; i < str.length; i++) {
    bytes.push(str.charCodeAt(i) & 0xff);
  }
  return bytes;
};

Original issue reported on code.google.com by toni.ruo...@gmail.com on 18 Jun 2010 at 8:57

[GoogleCodeExporter]

JavaScript characters use the Unicode character set, and charCodeAt can return 
values up to 65,535. If the string you're encoding has values above 255 and you 
bitwise AND those values with 0xFF, then you will be losing data.

So the things to consider are: 1) If your data is conceptually a series of 
characters, then you should probably use UTF8.stringToBytes rather than 
Binary.stringToBytes. Or 2), if your data is conceptually a series of bytes, 
then there shouldn't be any values higher than 255, and you should find out why 
there is.

Original comment by Jeff.Mott.OR on 18 Jun 2010 at 5:38

• Added labels: Type-Other
• Removed labels: Type-Defect

[GoogleCodeExporter]

We receive the data from XMLHttpRequest. The following tutorial shows that the 
data needs to be ANDed with 0xff...

https://developer.mozilla.org/en/using_xmlhttprequest#Receiving_binary_data

You could argue that we need to do that ourself because that specification asks 
us to do so. Or crypto-js could be helpful and accept the kind of strings XHR 
returns. What do you think? I thought crypto-js should do it because the 
application programmer might have problems controlling the part of character he 
is not interested in.

Original comment by toni.ruo...@gmail.com on 19 Jun 2010 at 8:07

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

So XHR gives me a byte string which is not clean. My options are: 1) write a 
janitor function which creates a new clean byte string 2) write my own function 
for turning an unclean byte string into a list of integers.

Option 1 is a bit tricky as javascript strings are immutable. Thus the 
operation can not be done in place. Option 2 leads to code duplication. Maybe 
the crypto-js function could have an option for taking a dirty byte string as 
input?

Original comment by toni.ruo...@gmail.com on 20 Jun 2010 at 11:23

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Fixed in revision 301. Will be included in release v2.0.1.

Original comment by Jeff.Mott.OR on 7 Mar 2011 at 4:04

• Changed state: Fixed
• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Original comment by Jeff.Mott.OR on 20 Jun 2010 at 9:11

• Changed state: Accepted
• Added labels: ****
• Removed labels: ****