nst/JSONTestSuite

"JavaScript" is too broad, needs more engines

Opened this issue · 0 comments

Even being the birthplace of JSON, JavaScript still has several implementations and engines which behave differently.

The parser labeled "JavaScript" uses NodeJS, whichever version is currently installed indiscriminately. I tested all of the currently used node versions (4 to 10) and they all behave similarly, but this is worth documenting at least.

I think it'd be a good idea to provide tests for major browser engines too. With WebKit/V8, it's definitely possible. I did tests with PhantomJS, but it's a poor choice for multiple reasons. Couldn't figure how to run Headless Chrome properly, but it should be doable. For other engines/browsers it's trickier and may even be impossible.

PhantomJS (2.1.1 with WebKit 538.1) results compared with NodeJS 9.6.1 are:

image

The problem with phantom is that it's abandoned, incredibly slow (couldn't even fit into 5 seconds timeout in tests) and iirc uses a JS engine different from V8.