brianmario/yajl-ruby

Allow encoding of already encoded JSON strings..

Opened this issue · 6 comments

pkieltyka commented

Imagine you have a string, that already has JSON.. but you'd like to wrap that JSON object in an array with others. You may have gotten that JSON object from a cache, or maybe its a Javascript function in a string (like a Handlebars pre-compiled template). It would be nice to effectively "join" JSON objects during encoding. This is a nice optimization, and also convenient. Below is an example and idea on how to implement this.

# One idea to implement this is....

# Get an array of json-encoded strings with ids 1, 2, 3, and 4
x = JsonCache.get([1,2,3,4]).map {|o| Yajl::JsonString.new(o) }

# x[0] = "{ id: 1, name: John}"
# x[1] = "{ id: 2, name: Peter}"
# x[2] = "{ id: 3, name: Mike}"
# x[3] = "{ id: 4, name: Allan}"

json = Yajl::Encoder.encode(x) 

# json should look like:
# "[{ id: 1, name: John}, { id: 2, name: Peter}, { id: 3, name: Mike}, { id: 4, name: Allan}]"

# currently Yajl will encode each string and instead will return:
# "[\"{ id: 1, name: John}\", \"{ id: 2, name: Peter}\", \"{ id: 3, name: Mike}\", \"{ id: 4, name: Allan}\"]"

Of course you could just decode each object in x, put it in an array, and then re-encode. Fine. But thats unnecessary, and here's a better example that can't be achieved unless the encoder understands strings as native JSON types to just concatenate them to the result.

# Pseudo class..
compiled_template = HandlebarsCompiler.compile("hi {{var}}")

# x
# => "function (Handlebars,depth0,helpers,partials,data) {\n  helpers = helpers || Handlebars.helpers;\n  var buffer = \"\", stack1, foundHelper, self=this, functionType=\"function\", helperMissing=helpers.helperMissing, undef=void 0, escapeExpression=this.escapeExpression;\n\n\n  buffer += \"hi \";\n  foundHelper = helpers['var'];\n  stack1 = foundHelper || depth0['var'];\n  if(typeof stack1 === functionType) { stack1 = stack1.call(depth0, { hash: {} }); }\n  else if(stack1=== undef) { stack1 = helperMissing.call(depth0, \"var\", { hash: {} }); }\n  buffer += escapeExpression(stack1);\n  return buffer;}"

template = {
  :name => Yajl::JsonString.new(compiled_template)
}

puts Yajl::Encoder.encode(template)

# The idea is the encoder would return: "{ name: function (Handlebars,depth0,helpers,partials,data) { ..etc. etc. } }"

Now, when parsed by the browser, it doesn't have the eval() the function.

I just came up with the Yajl::JsonString class, which would inherit from a String, and add a method called "is_json" set to true. This way when the encoder is traversing strings, it can test for is_json, and just concatenate instead of encode.

What do you think?

brianmario commented

Interesting idea, I'll see if I can explore this a little more in my yajl-ruby 2.0 stuff (there's a branch going already). For now, something like this should work in place of Yajl::JsonString in your example:

class JsonWrapper
  def initialize(json_string)
    @json_string = json_string
  end

  # yajl-ruby will check if this method exists and call it if so
  # then append the return value directly onto the output buffer as-is
  # this means that this method is assumed to be returning valid JSON
  def to_json
    @json_string
  end
end

But, let me know if otherwise ;)

pkieltyka commented

Thanks. The JsonWrapper worked. I tried to write a JsonString class using your example as so:

class JsonString < String
  def initialize(json_string)
    @json_string = json_string
    super(json_string)
  end

  def to_json
    @json_string
  end
end

However, this didn't seem to work, the encoder still encoded the string. How come? I see in yajl_ext.c that a to_json method is being defined for Strings.. and interestingly, the to_json method in JsonString isn't being called.. it's calling the to_json in String.

Either way, the JsonWrapper will get me by, thanks!

brianmario commented

Ah - this is because yajl-ruby will (for the sake of efficiency) handle encoding anything that directly translates to a native JSON type (string, number, float, true, false, nil, array and hash) directly down in C.

brianmario commented

Damn, didn't mean submit that yet...

Anyway, in those cases I don't check for to_json being defined. The to_json check is more of a fallback in case the object being encoded isn't one of those natively translatable types (like an ActiveRecord::Base instance for example). If the object doesn't response to to_json either, then I finally fall back to just calling to_s.

teamon commented

Any progress on that? My use case is encoding quite big arrays of complex objects and right now I ended up with cached objects as json string and creating the final json "by hand" just joining strings etc. I'd love to see yajl encoder that could handle that.