asyrjasalo/RESTinstance

Cannot validate properties with numeric name

tpokki opened this issue · 9 comments

Same problem with both 1.1.0 and 1.2.0.

Json:

{
  "element": {
      "1": "foobar",
      "2": "foobar",
      "3": "foobar"
  }
}    

Test 1:

    String                     $.element.1   foobar

Error 1:

Invalid JSONPath query '$.element.1': 'NoneType' object has no attribute 'lineno'

Test 2:

   String                     $.element['1']   foobar

Error 2:

Property '1' does not exist in:
{
    "1": "foobar",
    "2": "foobar",
    "3": "foobar"
}

This works:

    Object                     $.elements     required=["1", "2", "3"]  additionalProperties=false

Hi, thanks for the issue. Just to make this sure, does the error also come with response body element 1?

Yes, same error with that one:

Property '1' does not exist in:
{
    "1": "foobar",
    "2": "foobar",
    "3": "foobar"
}

Thanks, this definitely is a bug. I'll look into this as soon as I can.

Temporary workaround:

Should Be Equal As Strings  ${res.body['elements']['1']}   foobar

The JSONPath parsing is done with jsonpath-ng, which in turn gives the error. So, it seems like the bug is with jsonpath-ng instead of this library. I can see if there is something we can do [with a reasonable amount of work]. But in the meanwhile, consider opening a ticket to jsonpath-ng.

Now I did a double check. It seems that the problem with jsonpath-ng is the $.element.1 format and it parses $.element['1'] normally. That means that finding the value with the JSONPath is an error with this library. I will continue investigating, although I'm fairly confident I already found the cause for this.

There is issue h2non/jsonpath-ng#38 already opened for jsonpath-ng. I checked the code briefly, and it seems that the parser/grammar has as few flaws in it. It assumes that ID contain always non-numeric character, while numeric token in the language always implicates idx.

Tested also that adding grammar rule to fields -> NUMBER will make the $.element.1 work, but it will break grammar for jsonpath -> [ idx ].

Don't know enough of the underlying ply/yacc parser, and the jsonpath-ng to come up with proper change. That is, in my opinion to determine whether token is ID or NUMBER should rely on the context. If the token was preceded by . it would be considered to be ID, and in case of [ it would be NUMBER.

/work/jsonpath_ng# echo "$.element.1" | python lexer.py
< output omitted >
$                   $
.                   .
element             ID
.                   .
1                   NUMBER
/work/jsonpath_ng# echo "$.element.u1" | python lexer.py
< output omitted >
$                   $
.                   .
element             ID
.                   .
u1                  ID

Anyway, good if you come up with a way to fix this in RESTInstance.

@tpokki I fixed $.element['1'] and applied the fix to the Eficode fork of this repository. We'll merge the next release here once it's ready.

Closing this issue now as this has been fixed. We'll make a new release at the beginning of February. If other syntax than $.element['1'] wants to be used, please open another issue.