/JSONKit

Objective-C JSON

Primary LanguageObjective-C

JSONKit

JSONKit is licensed under the terms of the BSD License. Copyright © 2011, John Engelhart.

A Very High Performance Objective-C JSON Library

ParsingSerializing
Deserialize from JSON Serialize to JSON
23% Faster than Binary .plist !549% Faster than Binary .plist !
  • Benchmarking was performed on a MacBook Pro with a 2.66GHz Core 2.
  • All JSON libraries were compiled with gcc-4.2 -DNS_BLOCK_ASSERTIONS -O3 -arch x86_64.
  • Timing results are the average of 1,000 iterations of the user + system time reported by getrusage.
  • The JSON used was twitter_public_timeline.json from samsoffes / json-benchmarks.
  • Since the .plist format does not support serializing NSNull, the null values in the original JSON were changed to "null".
  • The experimental branch contains the gzip compression changes.
    • JSONKit automagically links to libz.dylib on the fly at run time– no manual linking required.
    • Parsing / deserializing will automagically decompress a buffer if it detects a gzip signature header.
    • You can compress / gzip the serialized JSON by passing JKSerializeOptionCompress to -JSONDataWithOptions:error:.

JavaScript Object Notation, or JSON, is a lightweight, text-based, serialization format for structured data that is used by many web-based services and API's. It is defined by RFC 4627.

JSON provides the following primitive types:

  • null
  • Boolean true and false
  • Number
  • String
  • Array
  • Object (a.k.a. Associative Arrays, Key / Value Hash Tables, Maps, Dictionaries, etc.)

These primitive types are mapped to the following Objective-C Foundation classes:

JSONObjective-C
nullNSNull
true and falseNSNumber
NumberNSNumber
StringNSString
ArrayNSArray
ObjectNSDictionary

JSONKit uses Core Foundation internally, and it is assumed that Core Foundation ≡ Foundation for every equivalent base type, i.e. CFString ≡ NSString.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

JSON To Objective-C Primitive Mapping Details

  • The JSON specification is somewhat ambiguous about the details and requirements when it comes to Unicode, and it does not specify how Unicode issues and errors should be handled. Most of the ambiguity stems from the interpretation and scope RFC 4627 Section 3, Encoding: JSON text SHALL be encoded in Unicode. It is the authors opinion and interpretation that the language of RFC 4627 requires that a JSON implementation MUST follow the requirements specified in the Unicode Standard, and in particular the Conformance chapter of the Unicode Standard, which specifies requirements related to handling, interpreting, and manipulating Unicode text.

    The default behavior for JSONKit is strict RFC 4627 conformance. It is the authors opinion and interpretation that RFC 4627 requires JSON to be encoded in Unicode, and therefore JSON that is not legal Unicode as defined by the Unicode Standard is invalid JSON. Therefore, JSONKit will not accept JSON that contains ill-formed Unicode. The default, strict behavior implies that the JKParseOptionLooseUnicode option is not enabled.

    When the JKParseOptionLooseUnicode option is enabled, JSONKit follows the specifications and recommendations given in The Unicode 6.0 standard, Chapter 3 - Conformance, section 3.9 Unicode Encoding Forms. As a general rule of thumb, the Unicode code point U+FFFD is substituted for any ill-formed Unicode encountered. JSONKit attempts to follow the recommended Best Practice for Using U+FFFD: Replace each maximal subpart of an ill-formed subsequence by a single U+FFFD.

    The following Unicode code points are treated as ill-formed Unicode, and if JKParseOptionLooseUnicode is enabled, cause U+FFFD to be substituted in their place:

    U+0000.
    U+D800 thru U+DFFF, inclusive.
    U+FDD0 thru U+FDEF, inclusive.
    U+nFFFE and U+nFFFF, where n is from 0x0 to 0x10

    The code points U+FDD0 thru U+FDEF, U+nFFFE, and U+nFFFF (where n is from 0x0 to 0x10), are defined as Noncharacters by the Unicode standard and "should never be interchanged".

    An exception is made for the code point U+0000, which is legal Unicode. The reason for this is that this particular code point is used by C string handling code to specify the end of the string, and any such string handling code will incorrectly stop processing a string at the point where U+0000 occurs. Although reasonable people may have different opinions on this point, it is the authors considered opinion that the risks of permitting JSON Strings that contain U+0000 outweigh the benefits. One of the risks in allowing U+0000 to appear unaltered in a string is that it has the potential to create security problems by subtly altering the semantics of the string which can then be exploited by a malicious attacker. This is similar to the issue of arbitrarily deleting characters from Unicode text.

    RFC 4627 allows for these limitations under section 4, Parsers: An implementation may set limits on the length and character contents of strings.

    It is important to note that JSONKit will not delete characters from the JSON being parsed as this is a requirement specified by the Unicode Standard. Additional information can be found in the Unicode Security FAQ and Unicode Technical Report #36 - Unicode Security Consideration, in particular the section on non-visual security.

  • The JSON specification does not specify the details or requirements for JSON String values, other than such strings must consist of Unicode code points, nor does it specify how errors should be handled. While JSONKit makes an effort (subject to the reasonable caveats above regarding Unicode) to preserve the parsed JSON String exactly, it can not guarantee that NSString will preserve the exact Unicode semantics of the original JSON String.

    JSONKit does not perform any form of Unicode Normalization on the parsed JSON Strings, but can not make any guarantees that the NSString class will not perform Unicode Normalization on the parsed JSON String used to instantiate the NSString object. The NSString class may place additional restrictions or otherwise transform the JSON String in such a way so that the JSON String is not bijective with the instantiated NSString object. In other words, JSONKit can not guarantee that when you round trip a JSON String to a NSString and then back to a JSON String that the two JSON Strings will be exactly the same, even though in practice they are. For clarity, "exactly" in this case means bit for bit identical. JSONKit can not even guarantee that the two JSON Strings will be Unicode equivalent, even though in practice they will be and would be the most likely cause for the two round tripped JSON Strings to no longer be bit for bit identical.

  • JSONKit maps true and false to the CFBoolean values kCFBooleanTrue and kCFBooleanFalse, respectively. Conceptually, CFBoolean values can be thought of, and treated as, NSNumber class objects. The benefit to using CFBoolean is that true and false JSON values can be round trip deserialized and serialized without conversion or promotion to a NSNumber with a value of 0 or 1.

  • The JSON specification does not specify the details or requirements for JSON Number values, nor does it specify how errors due to conversion should be handled. In general, JSONKit will not accept JSON that contains JSON Number values that it can not convert with out error or loss of precision.

    For non-floating-point numbers (i.e., JSON Number values that do not include a . or e|E), JSONKit uses a 64-bit C primitive type internally, regardless of whether the target architecture is 32-bit or 64-bit. For unsigned values (i.e., those that do not begin with a -), this allows for values up to 264-1 and up to -263 for negative values. As a special case, the JSON Number -0 is treated as a floating-point number since the underlying floating-point primitive type is capable of representing a negative zero, whereas the underlying twos-complement non-floating-point primitive type can not. JSON that contains Number values that exceed these limits will fail to parse and optionally return a NSError object. The functions strtoll() and strtoull() are used to perform the conversions.

    The C double primitive type, or IEEE 754 Double 64-bit floating-point, is used to represent floating-point JSON Number values. JSON that contains floating-point Number values that can not be represented as a double (i.e., due to over or underflow) will fail to parse and optionally return a NSError object. The function strtod() is used to perform the conversion. Note that the JSON standard does not allow for infinities or NaN (Not a Number).

  • For JSON Objects (or NSDictionary in JSONKit nomenclature), RFC 4627 says The names within an object SHOULD be unique (note: name is a key in JSONKit nomenclature). At this time the JSONKit behavior is undefined for JSON that contains names within an object that are not unique. However, JSONKit currently tries to follow a "the last key / value pair parsed is the one chosen" policy. This behavior is not finalized and should not be depended on.

    The previously covered limitations regarding JSON Strings have important consequences for JSON Objects since JSON Strings are used as the key. The JSON specification does not specify the details or requirements for JSON Strings used as keys in JSON Objects, specifically what it means for two keys to compare equal. Unfortunately, because RFC 4627 states JSON text SHALL be encoded in Unicode., this means that one must use the Unicode Standard to interpret the JSON, and the Unicode Standard allows for strings that are encoded with different Unicode Code Points to "compare equal". JSONKit uses NSString exclusively to manage the parsed JSON Strings. Because NSString is focused capatability with the Unicode Standard, there exists the possibility that NSString may subtly and silently convert the Unicode Code Points contained in the original JSON String in to a Unicode equivalent string. Due to this, the JSONKit behavior for JSON Strings used as keys in JSON Objects that may be Unicode equivalent but not binary equivalent is undefined.

Objective-C To JSON Primitive Mapping Details

  • The NSDictionary class allows for any object, which can be of any class, to be used as a key. JSON, however, only permits Strings to be used as keys. Therefore JSONKit will fail with an error if it encounters a NSDictionary that contains keys that are not NSString objects during serialization.

  • JSON does not allow for Numbers that are ±Infinity or ±NaN. Therefore JSONKit will fail with an error if it encounters a NSNumber that contains such a value during serialization.

  • JSONKit will fail with an error if it encounters an object that is not a NSNull, NSNumber, NSString, NSArray, or NSDictionary class object during serialization.

  • Objects created with [NSNumber numberWithBool:YES] and [NSNumber numberWithBool:NO] will be mapped to the JSON values of true and false, respectively.

Reporting Bugs

Please use the github.com JSONKit Issue Tracker to report bugs.

The author requests that you do not file a bug report with JSONKit regarding problems reported by the clang static analyzer unless you first manually verify that it is an actual, bona-fide problem with JSONKit and, if appropriate, is not "legal" C code as defined by the C99 language specification. If the clang static analyzer is reporting a problem with JSONKit that is not an actual, bona-fide problem and is perfectly legal code as defined by the C99 language specification, then the appropriate place to file a bug report or complaint is with the developers of the clang static analyzer.

Important Details

  • JSONKit is not designed to be used with the Mac OS X Garbage Collection. The behavior of JSONKit when compiled with -fobj-gc is undefined. It is extremely unlikely that Mac OS X Garbage Collection will ever be supported.

  • The JSON to be parsed by JSONKit MUST be encoded as Unicode. In the unlikely event you end up with JSON that is not encoded as Unicode, you must first convert the JSON to Unicode, preferably as UTF8. One way to accomplish this is with the NSString methods -initWithBytes:length:encoding: and -initWithData:encoding:.

  • Internally, the low level parsing engine uses UTF8 exclusively. The JSONDecoder method -parseJSONData: takes a NSData object as its argument and it is assumed that the raw bytes contained in the NSData is UTF8 encoded, otherwise the behavior is undefined.

  • It is not safe to use the same instantiated JSONDecoder object from multiple threads at the same time. If you wish to share a JSONDecoder between threads, you are responsible for adding mutex barriers to ensure that only one thread is decoding JSON using the shared JSONDecoder object at a time.

Tips for speed

  • Enable the NS_BLOCK_ASSERTIONS pre-processor flag. JSONKit makes heavy use of NSCParameterAssert() internally to ensure that various arguments, variables, and other state contains only legal and expected values. If an assertion check fails it causes a run time exception that will normally cause a program to terminate. These checks and assertions come with a price: they take time to execute and do not contribute to the work being performed. It is perfectly safe to enable NS_BLOCK_ASSERTIONS as JSONKit always performs checks that are required for correct operation. The checks performed with NSCParameterAssert() are completely optional and are meant to be used in "debug" builds where extra integrity checks are usually desired. While your mileage may vary, the author has found that adding -DNS_BLOCK_ASSERTIONS to an -O2 optimization setting can generally result in an approximate 7-12% increase in performance.

  • Since the very low level parsing engine works exclusively with UTF8 byte streams, anything that is not already encoded as UTF8 must first be converted to UTF8. While JSONKit provides additions to the NSString class which allows you to conveniently convert JSON contained in a NSString, this convenience does come with a price. JSONKit uses the -UTF8String method to obtain a UTF8 encoded version of a NSString, and while the details of how a strings performs that conversion are an internal implementation detail, it is likely that this conversion carries a cost both in terms of time and the memory needed to store the conversion result. Therefore, if speed is a priority, you should avoid using the NSString convenience methods if possible.

  • If you are receiving JSON data from a web server, and you are able to determine that the raw bytes returned by the web server is JSON encoded as UTF8, you should use the JSONDecoder method -parseUTF8String:length: which immediately begins parsing the pointers bytes. In practice, every JSONKit method that converts JSON to an Objective-C object eventually calls this method to perform the conversion.

  • If you are using one of the various ways provided by the NSURL family of classes to receive JSON results from a web server, which typically return the results in the form of a NSData object, and you are able to determine that the raw bytes contained in the NSData are encoded as UTF8, then you should use either the JSONDecoder method parseJSONData: or the NSData method -objectFromJSONData. If are going to be converting a lot of JSON, the better choice is to instantiate a JSONDecoder object once and use the same instantiated object to perform all your conversions. This has two benefits:

    1. The NSData method -objectFromJSONData creates an autoreleased JSONDecoder object to perform the one time conversion. By instantiating a JSONDecoder object once and using the parseJSONData: method repeatedly, you can avoid this overhead.
    2. The instantiated object cache from the previous JSON conversion is reused. This can result in both better performance and a reduction in memory usage if the JSON your are converting is very similar. A typical example might be if you are converting JSON at periodic intervals that consists of real time status updates.

  • On average, the JSONData… methods are nearly four times faster than the JSONString… methods when serializing a NSDictionary or NSArray to JSON. The difference in speed is due entirely to the instantiation overhead of NSString.

Parsing Interface

JSONDecoder Interface

Note: The bytes contained in a NSData object MUST be UTF8 encoded.

Important: Methods will raise NSInvalidArgumentException if parseOptionFlags is not valid.
Important: parseUTF8String: will raise NSInvalidArgumentException if parseUTF8String is NULL.
Important: parseJSONData: will raise NSInvalidArgumentException if jsonData is NULL.

+ (id)decoder;
+ (id)decoderWithParseOptions:(JKParseOptionFlags)parseOptionFlags;
- (id)initWithParseOptions:(JKParseOptionFlags)parseOptionFlags;

- (void)clearCache;

- (id)parseUTF8String:(const unsigned char *)string length:(size_t)length;
- (id)parseUTF8String:(const unsigned char *)string length:(size_t)length error:(NSError **)error;

- (id)parseJSONData:(NSData *)jsonData;
- (id)parseJSONData:(NSData *)jsonData error:(NSError **)error;

NSString Interface

- (id)objectFromJSONString;
- (id)objectFromJSONStringWithParseOptions:(JKParseOptionFlags)parseOptionFlags;
- (id)objectFromJSONStringWithParseOptions:(JKParseOptionFlags)parseOptionFlags error:(NSError **)error;

NSData Interface

- (id)objectFromJSONData;
- (id)objectFromJSONDataWithParseOptions:(JKParseOptionFlags)parseOptionFlags;
- (id)objectFromJSONDataWithParseOptions:(JKParseOptionFlags)parseOptionFlags error:(NSError **)error;

JKParseOptionFlags

Parsing OptionDescription
JKParseOptionNoneThis is the default if no other other parse option flags are specified, and the option used when a convenience method does not provide an argument for explicitly specifying the parse options to use. Synonymous with JKParseOptionStrict.
JKParseOptionStrictThe JSON will be parsed in strict accordance with the RFC 4627 specification.
JKParseOptionCommentsAllow C style // and /* … */ comments in JSON. This is a fairly common extension to JSON, but JSON that contains C style comments is not strictly conforming JSON.
JKParseOptionUnicodeNewlinesAllow Unicode recommended (?:\r\n|[\n\v\f\r\x85\p{Zl}\p{Zp}]) newlines in JSON. The JSON specification only allows the newline characters \r and \n, but this option allows JSON that contains the Unicode recommended newline characters to be parsed. JSON that contains these additional newline characters is not strictly conforming JSON.
JKParseOptionLooseUnicodeNormally the decoder will stop with an error at any malformed Unicode. This option allows JSON with malformed Unicode to be parsed without reporting an error. Any malformed Unicode is replaced with \uFFFD, or REPLACEMENT CHARACTER, as specified in The Unicode 6.0 standard, Chapter 3, section 3.9 Unicode Encoding Forms.
JKParseOptionPermitTextAfterValidJSONNormally, white-space that follows the JSON is interpreted as a parsing failure. This option allows for any trailing white-space to be ignored and not cause a parsing error.

Serializing Interface

Note: The bytes contained in the returned NSData object is UTF8 encoded.

NSArray and NSDictionary Interface

- (NSData *)JSONData;
- (NSData *)JSONDataWithOptions:(JKSerializeOptionFlags)serializeOptions error:(NSError **)error;
- (NSString *)JSONString;
- (NSString *)JSONStringWithOptions:(JKSerializeOptionFlags)serializeOptions error:(NSError **)error;

JKSerializeOptionFlags

Serializing OptionDescription
JKSerializeOptionNoneThis is the default if no other other serialize option flags are specified, and the option used when a convenience method does not provide an argument for explicitly specifying the serialize options to use.
JKSerializeOptionEscapeUnicodeWhen JSONKit encounters Unicode characters in NSString objects, the default behavior is to encode those Unicode characters as UTF8. This option causes JSONKit to encode those characters as \uXXXX. For example,
["w∈L⟺y(∣y∣≤∣w∣)"]
becomes:
["w\u2208L\u27fa\u2203y(\u2223y\u2223\u2264\u2223w\u2223)"]