joshbuddy/jsonpath

Getting TypeError if one of the nodes in json has name "hash"

clergyman opened this issue · 24 comments

clergyman commented

Hi @Skarlso!
I have a strange issue, suppose the following json:
j = {height: 5, hash: "some_hash"}.to_json
And the following paths
ht = JsonPath.new "$..height"
hs = JsonPath.new "$..hash"
Then in ruby 2.6.0 condole I see this - height is found and hash gives me TypeError exception:

2.6.0 :008 > ht.on j
=> [5]

2.6.0 :010 > hs.on j
Traceback (most recent call last):
16: from (irb):10
15: from (irb):10:in 'rescue in irb_binding'
14: from /Gems/ruby-2.6.0/gems/jsonpath-1.0.4/lib/jsonpath.rb:73:in 'on'
13: from /Gems/ruby-2.6.0/gems/jsonpath-1.0.4/lib/jsonpath.rb:73:in 'to_a'
12: from /Gems/ruby-2.6.0/gems/jsonpath-1.0.4/lib/jsonpath/enumerable.rb:28:in 'each'
11: from /Gems/ruby-2.6.0/gems/jsonpath-1.0.4/lib/jsonpath/enumerable.rb:26:in 'each'
10: from /Gems/ruby-2.6.0/gems/jsonpath-1.0.4/lib/jsonpath/enumerable.rb:39:in 'each'
9: from /Gems/ruby-2.6.0/gems/jsonpath-1.0.4/lib/jsonpath/enumerable.rb:39:in 'each'
8: from /Gems/ruby-2.6.0/gems/jsonpath-1.0.4/lib/jsonpath/enumerable.rb:39:in 'block in each'
7: from /Gems/ruby-2.6.0/gems/jsonpath-1.0.4/lib/jsonpath/enumerable.rb:30:in 'each'
6: from /Gems/ruby-2.6.0/gems/jsonpath-1.0.4/lib/jsonpath/enumerable.rb:60:in 'handle_wildecard'
5: from /Gems/ruby-2.6.0/gems/jsonpath-1.0.4/lib/jsonpath/enumerable.rb:60:in 'each'
4: from /Gems/ruby-2.6.0/gems/jsonpath-1.0.4/lib/jsonpath/enumerable.rb:68:in 'block in handle_wildecard'
3: from /Gems/ruby-2.6.0/gems/jsonpath-1.0.4/lib/jsonpath/enumerable.rb:22:in 'each'
2: from /Gems/ruby-2.6.0/gems/jsonpath-1.0.4/lib/jsonpath/enumerable.rb:134:in 'yield_value'
1: from /Gems/ruby-2.6.0/gems/jsonpath-1.0.4/lib/jsonpath/enumerable.rb:134:in '[]'
TypeError (no implicit conversion of String into Integer)

Can you please have a look at it if time?

Skarlso commented

Hi @clergyman! Wow that's interesting. I'll take a look as soon as I can! Thanks for reporting. :) I'm guessing hash as a word will have some trouble haha.

clergyman commented

Yess, this is highly possible )) I came upon this as I had to repair my old tests, and it appears that everything runs fine in ruby 2.3.3, so the trouble with keyword hash came in in later versions. But unfortunately that's an old stable product and we cannot change contracts in it, and anyway it looks like an easy although kind of ugly fix.

Skarlso commented
Skarlso commented

@clergyman Hi.

So I just tried this on ruby 2.6 and both the unit test and irb is working fine. :D

 Mon 14 Oct 09:38:30 ❯ ~/RubyProjects/jsonpath ❯  master ✘ ❯ gbrautigam ❯
 $ be irb
2.6.0 :001 > require 'jsonpath'
 => true
2.6.0 :002 > require 'json'
 => true
2.6.0 :003 > j = {height: 5, hash: "some_hash"}.to_json
 => "{\"height\":5,\"hash\":\"some_hash\"}"
2.6.0 :004 > ht = JsonPath.new "$..height"
 => #<JsonPath:0x00007fa1e5186030 @opts={}, @path=["$", "..", "['height']"]>
2.6.0 :005 > hs = JsonPath.new "$..hash"
 => #<JsonPath:0x00007fa1e519c3f8 @opts={}, @path=["$", "..", "['hash']"]>
2.6.0 :006 >

  def test_hash_keyname
    j = {height: 5, hash: "some_hash"}.to_json

    ht = JsonPath.new "$..height"
    hs = JsonPath.new "$..hash"
    p ht
    p hs
  end

What am I doing wrong?

clergyman commented

@Skarlso JsonPath object is created fine, have you tried further?

ht.on j
hs.on j

I have the whole thing crash on hs.on j

Skarlso commented 
2.6.0 :004 > j = {height: 5, hash: "some_hash"}.to_json
 => "{\"height\":5,\"hash\":\"some_hash\"}"
2.6.0 :005 > hs = JsonPath.new "$..hash"
 => #<JsonPath:0x00007fd2999697e0 @opts={}, @path=["$", "..", "['hash']"]>
2.6.0 :006 > hs.on j
...
TypeError (no implicit conversion of String into Integer)
2.6.0 :007 >

?

clergyman commented

@Skarlso yep, that's it, ["some_hash"] expected, but exception =(

Skarlso commented

Uh, this is disconcerting. They look the same in JsonPath as an object.

Skarlso commented

Okay, interesting. It's not the word hash apparently.

{"height"=>5, "fake"=>"some_hash"}
Skarlso commented

Oh Gods it's the path...

hs = JsonPath.new "$..fake"

Works...

hs = JsonPath.new "$..hash"

doesn't work.

clergyman commented

The path does not work if the node name is hash. And yes this is amazing ))))

Skarlso commented

Nope. It doesn't matter what the node name is. Sorry, I wasn't clear on that. Just the PATH doesn't work if the path to look for contains the word hash. So the json could be:

{"height"=>5, "fake"=>"some_hash"}

Or

{"height"=>5, "hash"=>"some_hash"}

Doens't matter as long as the path is $..hash. :D

clergyman commented

Oh I see.. Even more interesting!

Skarlso commented

@clergyman !!!!!!!!

2.6.0 :002 > 5.respond_to?(:hash)
 => true

F*cking ruby.

  • 😄1
Skarlso commented

Freck. No idea how to fix this without hacking. :(

Skarlso commented

Alright, I think I have an idea.. I can't fix ruby, but at least I can ward against invalid output somewhat... 🤔

clergyman commented

Good Lord, i didn't really expect this is such a crazy stuff..

Skarlso commented

So the problem is that in order to use structs too there is a part where we check if the result responds to a function call.

This part:

elsif node.respond_to?(k.to_s)

Which means that anything you write in there that the basic Ruby Object responds to will cause this problem. hash is one. Another would be send and all of these:

https://ruby-doc.org/core-2.6.5/Object.html

Which is a problem... :)

My solution is to have a check which will not do this in case of built-in types. So only custom types will be checked for respond_to.

Skarlso commented

@clergyman Alright! I think I've got a good solution. :) Well, this was a nice ride. :D

clergyman commented

@Skarlso Yes, looks good, I'll check this in master, after that will you be so kind and release new version of gem with this fix?

Skarlso commented

Sure

clergyman commented

@Skarlso i tested in master, it's OK now, thank you so very much. I think let's close this one and create a new gem release?

Skarlso commented

Sure. :0

Skarlso commented

@clergyman Done. :) 1.0.5 released. :)