bterlson/test262-harness

exception names missing in node (not node-ip) runner >= 0.11 (incl 0.12, io.js )

smikes opened this issue · 3 comments

When I use node 0.10, test262-harness works. When I use 0.11, 0.12, or io.js, this test fails because instead of TypeError I'm getting evalmachine.<anonymous>:82 ; this only seems to happen with the out-of-process node runner, though.

./tools/packaging/test262.py --command 'node' 11.13.1-4-28
language/expressions/assignment/11.13.1-4-28-s passed in strict mode
language/expressions/assignment/11.13.1-4-28gs failed in strict mode as expected

Svarog:test262 smikes$ node ../test262-harness/bin/run.js test/language/expressions/assignment/11.13.1-4-28*
FAIL test/language/expressions/assignment/11.13.1-4-28gs.js
     Strict Mode - TypeError is thrown if the identifier 'Math.PI' appears as the LeftHandSideExpression of simple assignment(=)
 (Strict Mode)
     Exp: error matching /TypeError/
     Got: evalmachine.<anonymous>:82

Ran 2 tests
1 passed
1 failed
Took 0.21 seconds
Svarog:test262 smikes$ nvm use 0.10
Now using node v0.10.35
Svarog:test262 smikes$ node ../test262-harness/bin/run.js test/language/expressions/assignment/11.13.1-4-28*
Ran 2 tests
2 passed
0 failed
Took 0.12 seconds
Svarog:test262 smikes$ 

Looking into this, it's probably related to actual test failures under node >= 0.11:

$ nvm use 0.10 ; npm test 
... 
1..201
# tests 201
# pass  201

# ok
$ nvm use 1.2; npm test
> test262-harness@1.3.1 test /Users/smikes/src/github/test262-harness
> tape test/test.js

TAP version 13
# -r node
ok 1 No stderr is present
ok 2 7 tests run
ok 3 test/collateral/async.js passes
ok 4 test/collateral/asyncNegative.js passes
not ok 5 test/collateral/bothStrict.js passes
  ---
    operator: equal
    expected: true
    actual:   false
    at: ChildProcess.<anonymous> (/Users/smikes/src/github/test262-harness/test/utils.js:84:9)
  ...
not ok 6 test/collateral/strict.js passes in strict mode
  ---
    operator: equal
    expected: true
    actual:   false
    at: ChildProcess.<anonymous> (/Users/smikes/src/github/test262-harness/test/utils.js:84:9)
  ...
ok 7 test/collateral/noStrict.js passes
ok 8 test/collateral/error.js fails
ok 9 Error name is Error
ok 10 Error message is failure message
ok 11 test/collateral/thrownError.js fails
not ok 12 Error name is Error
  ---
    operator: equal
    expected: 'Error'
    actual:   'evalmachine.<anonymous>:70'
    at: ChildProcess.<anonymous> (/Users/smikes/src/github/test262-harness/test/utils.js:84:9)
  ...
not ok 13 Error message is failure message
  ---
    operator: equal
    expected: 'failure message'
    actual:   undefined
    at: ChildProcess.<anonymous> (/Users/smikes/src/github/test262-harness/test/utils.js:84:9)
  ...
/Users/smikes/src/github/test262-harness/test/utils.js:48
                        t.ok(act.errorStack.indexOf(exp.topOfStack) > -1, "Sta
                                           ^
TypeError: Cannot read property 'indexOf' of undefined
    at /Users/smikes/src/github/test262-harness/test/utils.js:48:44
    at Array.forEach (native)
    at /Users/smikes/src/github/test262-harness/test/utils.js:38:29
    at ChildProcess.<anonymous> (/Users/smikes/src/github/test262-harness/test/utils.js:84:9)
    at emitTwo (events.js:85:13)
    at ChildProcess.emit (events.js:153:7)
    at Process.ChildProcess._handle.onexit (child_process.js:1044:12)
npm ERR! Test failed.  See above for more details.
anba commented

#28 should fix this.

Fixed, thanks very much!