bterlson/test262-harness

False negative when tests are run under a debugger

Opened this issue · 0 comments

Node (see nodejs/node#34799) emits console output when running tests under a debugger (e.g. VS Code's IDE debugger).

Debugger attached.\n is emitted at the beginning of the test, and Waiting for the debugger to disconnect...\n is emitted at the end.

test262-harness reports this as a test failure. I assume this is because "no console output" is considered a success while "some output" is considered a failure?

We can work around it (e.g. tc39/proposal-temporal#1812) but it might be better to work around it here.

Here's the code I've proposed for Temporal's workaround:

    if (result.message === 'Debugger attached.\nWaiting for the debugger to disconnect...\n') {
      // work around https://github.com/nodejs/node/issues/34799 when running tests in the debugger
      result.message = '';
      result.pass = true;
    }