uri-templates/uritemplate-test

Negative Test Cases

Closed this issue · 5 comments

damnhandy commented

Section 3 states that a URI Template process should stop processing a template when it encounters an expression that does not match URI template grammar. I have a fews of tests in mind but they don't really fit nicely into the current framework.

Given the test expression

 [ "{/keys", "???" ]

Obviously we're looking for a template processor to fail when there is no closing } but I'm not sure how this should be expressed in a test case. Given that these tests are optional according to the spec, I'm not a 100% sure that they'd be applicable to the standard test suite. At the very least, I'd envision them being optional.

mnot commented

I can think of three ways:

  1. Have a file "negative-tests.json", in the same format, where they're all expected to fail.
  2. Add a property to the test suite (like "level") called something like "expected".
  3. use false to denote an expected failure; e.g., [ "{/keys", false ]

I kind of like 3... Note that whatever we do, we'd probably put these in a separate file, as spec-examples is just that, and I'd like to see us gather all kinds of tests here; no need to make it a single-file suite (that way, people can select the tests they want by choosing files).

damnhandy commented

I was already going down the path of number 3 myself, so let's do that! :) I completely agree that these tests belong in a separate file given that reporting errors is not a MUST. I'll send a pull request later today.

hannesg commented

@damnhandy: +1

I'll rework my uritemplate library over the weekend and I could really use more tricky invalid examples.

damnhandy commented

I've issued a pull request for both issues #3 and #4. Regarding the negative test cases, I think it might get a bit more complicated. If you take a look at the last two tests, they likely should not fail, but they are also not valid URI templates. For example, the very last test uses the URI routing syntax found in frameworks like express.js:

   /vars/:var

A URI template processor like shouldn't bail on this test, citing invalid syntax, but one also shouldn't expect a properly expanded URI either. I guess my point is that we may need another category whereby the expected result isn't a just true or false, but that the expected result should not match. So I am thinking that we need something like option 2 to indicate that these tests will parse properly but will not yield the expected result. Thoughts?

mnot commented

Looks done (sure we'll have more).