sebv/node-wd-sync

When using Mocha, test assertion errors are not processed as expected

n1k0 opened this issue · 4 comments

n1k0 commented

I'm trying to use wd-sync with mocha v1.0.1 but it seems test failures are not triggered correctly:

// case_test.js
/* global describe, it */
/* jshint expr:true */

var assert = require('assert');
var wdSync = require('wd-sync');
var client = wdSync.remote();
var browser = client.browser;
var sync = client.sync;

describe("test case", function() {
  it("should pass this test", function(done) {
    this.timeout(10000); // browser start time
    sync(function() {
      browser.init({browserName: 'firefox'});
      browser.get('http://google.com/');
      assert(false);
      browser.quit();
      done();
    });
  });
});

Gives:

$ mocha test/functional/case_test.js

  AssertionError: false == true
    at Object.<anonymous> (/path/to/project/test/functional/case_test.js:16:7)
    at _sync (/path/to/project/node_modules/wd-sync/lib/wd-sync.js:95:21)
    at sync (/path/to/project/node_modules/wd-sync/node_modules/make-sync/lib/make-sync.js:133:14)

As you can see the error is caught and processed by the sync function while it should be by mocha itself.

A working workaround is to patch mocha's it() function:

function _it(description, cb) {
  return it(description, function(done) {
    var context = this;
    sync(function() {
      try {
        cb.call(context, done);
      } catch (err) {
        if (!done)
          throw err;
        browser.quit();
        done.call(context, err);
      }
    });
  });
}

describe("test case", function() {
  _it("should pass this test", function(done) {
    this.timeout(10000); // browser start time
    browser.init({browserName: 'firefox'});
    browser.get('http://google.com/');
    assert(false);
    browser.quit();
    done();
  });
});

which gives:

$ mocha test/functional/case_test.js

  ․

  ✖ 1 of 1 test failed:

  1) test case should pass this test:

  AssertionError: false == true
      at Context.<anonymous> (/Users/niko/Sites/talkilla/test/functional/case_test.js:32:5)
      at Object.<anonymous> (/Users/niko/Sites/talkilla/test/functional/case_test.js:15:12)
      at _sync (/Users/niko/Sites/talkilla/node_modules/wd-sync/lib/wd-sync.js:95:21)
      at sync (/Users/niko/Sites/talkilla/node_modules/wd-sync/node_modules/make-sync/lib/make-sync.js:133:14)

Though I'm finding the hook quite unpleasant and fragile.

sebv commented

The pull request does not seem to solve the issue.

zah commented

What are the problems you are still experiencing?
We've been using the fix for a couple of days now. I'm assuming that the Mocha tests are written using the wrap function as demonstrated in the samples.

sebv commented

Yes the fix works when using wrap but the initial post scope was wider, am wondering if the fix should be moved at the sync level.

sebv commented

No the problem is sync and mocha done callback integration, I don't see a generic way to do it at sync level, the recommended solution is to use wrap, it will hide the done concept totally, and with @zah PR, the exception handling should now work properly.