You should include the module is-my-json-valid by mafintosh, It's the fastest

Question

You should include the module is-my-json-valid by mafintosh, It's the fastest

Closed this issue 10 years ago · 21 comments

is-my-json-valid is the fastest JSON-schema validator around.
It blows the competition out of the water in speed.
There is a blog post about it here:
http://cosmicrealms.com/blog/2014/08/29/benchmark-of-node-dot-js-json-validation-modules-part-3/

Answer 1 · 2015-01-18T10:39:44.000Z

I mean in your benchmarks

Answer 2 · 2015-01-18T18:03:06.000Z

If the validator doesn't support the full spec, it is not a JSON Schema validator. I can write a very fast validator that doesn't support all of JSON Schema:

is_valid = (schema, document) ->
  true

That's a reductio ad absurdum, obviously. But the point should be clear. If a validator doesn't reject documents that are invalid according to the JSON Schema standard, there can be no valid (pun intended) benchmarking of it in comparison to actual JSON Schema implementations.

Answer 3 · 2015-01-18T18:05:06.000Z

I do want to review is-my-json-valid, though. And we should add to the benchmarks a check that the tested libraries can actually detect invalid documents.

Answer 4 · 2015-01-18T18:33:34.000Z

Actually I think it does the spec.

—
Sent from Mailbox

On Sun, Jan 18, 2015 at 7:05 PM, Matthew King notifications@github.com
wrote:

I do want to review is-my-json-valid, though. And we should add to the benchmarks a check that the tested libraries can actually detect invalid documents.

Reply to this email directly or view it on GitHub:
#72 (comment)

Answer 5 · 2015-01-18T21:55:25.000Z

is-my-json-valid should support all of schema v4 (except for refRemote, ref, which this validator doesn't seem to support either)

Answer 6 · 2015-01-19T06:06:26.000Z

It will be interesting to see how you handle this. You have a chance to really shine for the open source community.

Answer 7 · 2015-01-20T20:43:37.000Z

is-my-json-valid should support all of schema v4 (except for refRemote, ref, which this validator doesn't seem to support either)

Good deal. No, JSCK doesn't do refRemote yet.

Answer 8 · 2015-01-20T21:05:01.000Z

@mafintosh, I'm seeing some indications that is-my-json-valid does not, in fact, support all of JSON Schema draft 4. Feel free to email me if you want to discuss this in private.

Answer 9 · 2015-01-20T21:11:01.000Z

@automatthew we can do it in public. which parts are missing?

Answer 10 · 2015-01-20T21:21:36.000Z

I don't see where, for instance, "multipleOf" is implemented.

Following that line of inquiry, I changed the value of that keyword in your test fixture to something that should cause failures for both the valid and invalid documents. All tests still pass.

Answer 11 · 2015-01-21T22:05:11.000Z

sorry, we're going to pass on this. we can give you a shout-out in the readme or something.

is-my-json-valid is very fast but it doesn't seem to properly support the spec.

lots more detail here: https://www.pandastrike.com/posts/20150121-the-fast-and-the-spurious

Answer 12 · 2015-01-21T23:45:12.000Z

It will be interesting to see how you handle this. You have a chance to really shine for the open source community.

I found it very interesting.

Answer 13 · 2015-01-22T00:18:04.000Z

Didn't know there was a json-schema test-suite. I'll make is-my-json-valid pass that. The orderly thing isn't used for anything internally (or has anything to do with the performance). When I researched json-schema initially i just liked the syntax but since I don't really use it anymore I'll probably remove it at some point.

Answer 14 · 2015-01-22T00:27:25.000Z

You may also find these documents helpful:

http://tools.ietf.org/html/draft-zyp-json-schema-04

http://tools.ietf.org/html/draft-fge-json-schema-validation-00

Answer 15 · 2015-01-22T01:28:49.000Z

I took out most of the stuff about Orderly. Wasn't quite sure how that worked anyway. The good news is every time I tried it, I got [object Object] from toString(), so index.js was probably falling through to the catch every time. So everything I said about Orderly may have been irrelevant. The bad news is I got this result while trying to get is-my-json-valid to tell me that an invalid doc was valid, and that I got that to happen pretty quickly.

Answer 16 · 2015-01-22T09:25:07.000Z

@gilesbowkett @automatthew is-my-json-valid 2.0.0 now passes the entire JSONSchema draft4 test suite (with exception of remote refs and multibyte unicode string with min/max length)

Running your benchmark yields the following result

## Benchmarks for Draft 4

Schema: 'Event - Valid document'.  A simple schema, exercising very few attributes
Sample size: 64
Validations per sample: 1024

  JSCK
  Warming up: ................................
  Iterations: ................................................................

  tv4
  Warming up: ................................
  Iterations: ................................................................

  jayschema
  Warming up: ................................
  Iterations: ................................................................

  is-my-json-valid
  Warming up: ................................
  Iterations: ................................................................

  z-schema
  Warming up: ................................
  Iterations: ................................................................


  JSCK: validations/millisecond
  median: 228.955    max: 254.094    min: 143.277

  tv4: validations/millisecond
  median: 125.429    max: 133.629    min: 82.071

  jayschema: validations/millisecond
  median: 1.933    max: 2.08    min: 1.554

  is-my-json-valid: validations/millisecond
  median: 1528.358    max: 2348.624    min: 965.127

  z-schema: validations/millisecond
  median: 97.789    max: 100.53    min: 63.175

Relative speeds:
is-my-json-valid : 1.000
JSCK : 6.675
tv4 : 12.185
z-schema : 15.629
jayschema : 790.847


Schema: 'Configuration'.  A moderately complex schema with some nesting and value constraints
Sample size: 64
Validations per sample: 256

  JSCK
  Warming up: ................................
  Iterations: ................................................................

  tv4
  Warming up: ................................
  Iterations: ................................................................

  jayschema
  Warming up: ................................
  Iterations: ................................................................

  is-my-json-valid
  Warming up: ................................
  Iterations: ................................................................

  z-schema
  Warming up: ................................
  Iterations: ................................................................


  JSCK: validations/millisecond
  median: 107.113    max: 127.745    min: 52.502

  tv4: validations/millisecond
  median: 36.626    max: 46.486    min: 12.066

  jayschema: validations/millisecond
  median: 1.017    max: 1.127    min: 0.703

  is-my-json-valid: validations/millisecond
  median: 563.877    max: 992.248    min: 503.937

  z-schema: validations/millisecond
  median: 43.866    max: 45.674    min: 27.465

Relative speeds:
is-my-json-valid : 1.000
JSCK : 5.264
z-schema : 12.855
tv4 : 15.395
jayschema : 554.556


Schema: 'Transaction'.
Sample size: 64
Validations per sample: 64

  JSCK
  Warming up: ................................
  Iterations: ................................................................

  tv4
  Warming up: ................................
  Iterations: ................................................................

  jayschema
  Warming up: ................................
  Iterations: ................................................................

  is-my-json-valid
  Warming up: ................................
  Iterations: ................................................................

  z-schema
  Warming up: ................................
  Iterations: ................................................................


  JSCK: validations/millisecond
  median: 15.885    max: 16.763    min: 14.408

  tv4: validations/millisecond
  median: 3.541    max: 3.632    min: 3.004

  jayschema: validations/millisecond
  median: 0.046    max: 0.05    min: 0.041

  is-my-json-valid: validations/millisecond
  median: 69.945    max: 87.075    min: 38.186

  z-schema: validations/millisecond
  median: 4.068    max: 4.455    min: 2.156

Relative speeds:
is-my-json-valid : 1.000
JSCK : 4.403
z-schema : 17.193
tv4 : 19.756
jayschema : 1519.454

is-my-json-valid is still consistently ~5x faster than JSCK

Answer 17 · 2015-01-22T11:04:47.000Z

It will be interesting to see how you handle this. You have a chance to really shine for the open source community.

I found it very interesting.

It still is :-) But I'm really glad that you helped mafintosh make is-my-json-valid pass the tests.

Answer 18 · 2015-01-23T20:26:32.000Z

reopened because #74, and well done @mafintosh.

just FYI, the pandastrike.com blog post about this will probably get updated accordingly.

Answer 19 · 2015-01-24T16:38:29.000Z

@gilesbowkett cool! looking forward to reading it

Answer 20 · 2015-01-26T17:49:37.000Z

here you go: https://www.pandastrike.com/posts/20150123-congrats-imjv

Still got to add an update to the pandastrike.com post about IMJV, but my personal blog post was updated with a link.

Answer 21 · 2015-01-29T00:13:17.000Z

resolved in 5dba843