node-test
node-test
is a simple, asynchronous test runner. It's designed to address what are limitations of existing Node.js test runners. Writing tests should be just like writing any other code.
Table of Contents
- Core Philosophy
- Comparison With Other Test Runners
- Install
- Example
- API
- Running Multiple Suites
- Custom Reporters
- Todo
Core Philosophy
- Fast - Concurrent Tests
- No global variables
- Optional CLI - Running a test file directly (
node test/test-something.js
) produces the same output as using the CLI. - Minimal - Just runs your tests and gives you the results. There's no fancy error interpretation, just a plain old Node.js error callstack.
- No planning tests or counting assertions.
- Asynchronous Tests - Prefers Promises, but supports Callbacks
- Built-In assertion library build on the core
assert
module
Comparison With Other Test Runners
These test runners have many great features and heavily inspired this module. However, there are key differences.
Mocha, Jasmine
- Tests are run serially (one at a time).
- Defaults to using global variables (
describe()
,it()
, etc). - Mocha CLI is required, directly running a mocha test files fails.
tape
- Tests are run serially (one at a time).
- The number of assertions has to be manually counted and planned (
t.plan()
) for every test.
ava
- Ava CLI is required, so directly running a test files fails.
- Ava's stack trace interpreter often removes helpful information, accesses properties on your object (causing side effects), and can interfere with other libraries built in error reporting (ie. Long Stack Traces)
Install
$ npm install --save-dev node-test
Example
'use strict';
const Suite = require('node-test');
const suite = new Suite('My Suite Name');
suite.test('Test 1', t => {
return funcReturnsPromise()
.then(result => {
t.equal(result, 2);
});
});
suite.skip('Test 2', t => {
throw new Error('skipped');
});
suite.todo('Test 3 - Coming Soon');
suite.test('Test 4', (t, state, done) => {
funcWithCallback(done);
});
suite.failing('Test 5 - Need to fix this', t => {
t.equal(1+1, 3);
});
Output:
----My Suite Name----
pass 1 - Test 1
skip 2 - Test 2
todo 3 - Test 3 - Coming Soon
pass 4 - Test 4
fail 5 - Test 5 - Need to fix this #failing test
Total: 3
Failed: 1 (5)
Passed: 2 67%
Process finished with exit code 0
API
Suite
The first thing you do with node-test
is create a suite for your tests.
new Suite(name, [options])
- suite constructor
Arguments
name
: string - title for testoptions
: objectfailFast
: boolean (default: false) - if a single test fails, stop all remaining tests in the suite
const Suite = require('node-test');
const suite = new Suite('My Suite Name');
Concurrent Tests
By default node-test
runs tests concurrently. Concurrency means that the tests must be atomic. They should not depend on other tests for state.
The following methods are used to create concurrent tests.
suite.test(name, action)
- Create a new test.
Arguments
name
: string - title for testaction
: function(t, state, done) - test implementationt
: object (built-in assertions)- (optional)
state
: object - result ofbeforeEach
hook - (optional)
done
: function - callback for asynchronous tests validateError
: function - function to validate the error
Test Resolution
- Any
Error
throw synchronously will cause the test to fail. - Asynchronous test can return a Promise or use the
done()
callback.- If the test returns a Promise, the test will pass or fail if the promise resolves or rejects, respectively.
- If the
done()
callback is used, the test will fail if the first argument is defined. See Node.js style callbacks.
validateError
can turn a failing test into a passing test, if the error validates. (see below)
Synchronous Test:
suite.test('My Test', t => {
const result = funcReturnsNumber();
t.equal(result, 2);
});
Asynchronous Test with Promises:
suite.test('My Test', t => {
return funcReturnsPromise()
.then(result => {
t.equal(result, 2);
});
});
Asynchronous Tests with a Callback
suite.test('My Test', (t, state, done) => {
funcWithCallback((err, result) => {
t.noError(err);
t.equal(result, 2);
});
});
suite.test('My Test 2', (t, state, done) => {
funcWithCallbackNoValue(done);
});
Error Validation
validateError
tests that a specific error was throw. This is makes it easy to test to test error conditions. The test will pass, if validateError
is passed and does not throw.
Passing Test
t.throws(() => {
return Promise.reject(new Error('expected error'));
},
err => {
t.true(err instanceof Error);
t.equal(err.message, 'expected error');
});
Failing Test
t.throws(() => {
return Promise.reject(new Error('expected error'));
},
err => {
t.equal(err.message, 'other error');
});
suite.skip(name, action)
- Creates a new test that will be skipped when the suite is run.
Arguments
Same as suite.test()
.
suite.only(name, action)
- Creates a new test that will be run exclusively.
When the suite run, only
tests will be run and all other tests will be skipped.
Arguments
Same as suite.test()
.
suite.todo(name)
- Creates a test placeholder.
The "test" will be shown in the output, but will not count toward the total.
Arguments
name
: string - title for test
suite.failing(name, action)
- Creates a new test that is expected to fail.
The test included as a failed test in the output, but don't cause the exit code to change (indicating the build failed).
Arguments
Same as suite.test()
.
Serial Tests
node-test
also supports serial tests (one at a time). Most of the time concurrent test are preferable, however there are times (such as when accessing a database) that you may want tests to run serially.
Within a suite, serial tests are executed first followed by any concurrent tests. However, multiple suites are run concurrently. So two serial tests could run at the same time if they are members of different suites.
The following methods are used to create serial tests. There usage is identical to the equivalent concurrent method.
suite.serial.test(name, action)
Arguments
Same as suite.test()
.
suite.serial.skip(name, action)
Arguments
Same as suite.test()
.
suite.serial.only(name, action)
Arguments
Same as suite.test()
.
suite.serial.todo(name)
Arguments
Same as suite.test()
.
suite.serial.failing(name, action)
Arguments
Same as suite.test()
.
Hooks
node-test
provides four hooks. The hooks are run serially to the tests.
suite.before(action)
- Run before all tests in the suite.
Arguments
name
: string - title for testaction
: function(t, state, done) - test implementationt
: object (built-in assertions)- (optional)
done
: function - callback for asynchronous tests
suite.after(action)
- Run after all tests in the suite.
Arguments
Same as suite.before()
.
suite.beforeEach(action)
- Run before each individual tests in the suite.
Arguments
Same as suite.before()
.
The beforeEach
hook runs before each test in the suite.
suite.beforeEach()
Usage with suite.beforeEach(t => {
return {
data: 2
};
});
suite.test('My Test 1', (t, state) => {
t.equal(1+1, state.data);
state.date = 3;
});
suite.test('My Test 2', (t, state) => {
t.equal(2, state.data); //state is still 2, because beforeEach is run individual for each test
});
suite.afterEach(action)
- Run after each individual tests in the suite.
Arguments
Same as suite.before()
.
Other Members
suite.setTimeout(delay)
- Set the time limit for tests (default: 5000)
Tests whose execution time exceeds the delay will fail.
Arguments
delay
: number - timeout in milliseconds
t (Built-In Assertion Library)
node-test
includes an assertion library that is a bit more feature rich than the core assert module.
- For every method,
message
is optional. If defined, it will be displayed if the assertion fails.*
t.pass()
t.pass();
t.fail([message])
t.fail();
t.true(value, [message])
An assertion that value
is strictly true.
const value = true;
t.true(value);
const arr = ['a'];
t.true(arr.length === 1);
t.false(value, [message])
An assertion that value
is strictly false.
const value = false;
t.false(value);
t.truthy(value, [message])
alias: t.assert()
An assertion that value
is strictly truthy.
const value = 1;
t.truthy(value);
t.falsey(value, [message])
An assertion that value
is strictly falsey.
const value = 0;
t.falsey(value);
t.equal(value, expected, [message])
aliases: t.is(), t.equals()
An assertion that value
is strictly equal to expected
.
const value = 1;
t.equal(value, 1);
t.notEqual(value, expected, [message])
aliases: t.not(), t.notEquals()
An assertion that value
is strictly not equal to expected
.
const value = 1;
t.notEqual(value, 2);
t.deepEqual(value, expected, [message])
An assertion that value
is strictly and deeply equal to expected
. Deep equality is tested by the not-so-shallow module.
const value = { data: 1234 };
t.deepEqual(value, { data: 1234 });
t.notDeepEqual(value, expected, [message])
An assertion that value
is strictly and deeply not equal to expected
.
const value = { data: 1234 };
t.notDeepEqual(value, { data: 5678 });
t.greaterThan(value, expected, [message])
An assertion that value
is greater than expected
.
const value = 2;
t.greaterThan(value, 1);
t.greaterThanOrEqual(value, expected, [message])
An assertion that value
is greater than or equal to expected
.
const value = 2;
t.greaterThanOrEqual(value, 2);
t.lessThan(value, expected, [message])
An assertion that value
is less than expected
.
const value = 1;
t.lessThan(value, 2);
t.lessThanOrEqual(value, expected, [message])
An assertion that value
is less than or equal to expected
. *message
is optional. If defined, it will be displayed if the assertion fails.
const value = 1;
t.lessThanOrEqual(value, 1);
t.noError(error, [message])
An assertion that error
is falsey. This is functionally similar to t.falsey()
, but the assertion error message indicates the failure was due to an Error
.
funcWithCallback((err, result) => {
t.noError(err);
});
t.notThrows(fn, [message])
An assertion that fn
is function that does not throws an Error synchronously nor asynchronously (via Promise or callback).
Arguments
fn
: function([done]) - code to assert throws- (optional)
done
: function - callback for asynchronous test
- (optional)
Synchronous Assertion
Would Pass:
t.notThrows(() => {
t.equal(1, 1);
});
Would Fail:
t.throws(() => {
throw new Error('error');
});
Asynchronous Assertion with Promises
t.notThrows(() => {
return funcReturnsPromise();
});
Asynchronous/Synchronous Assertion with Callback
The callback can be asynchronous or synchronous. The callback can be executed immediately or later. It will be handled the same.
t.notThrows(done1 => {
funcWithAsyncCallback(done1);
});
t.notThrows(done1 => {
funcWithSyncCallback(done1);
});
t.notThrows(done1 => {
funcWithCallback((err, result) => {
t.noError(err);
t.equal(result, 2);
done1();
});
});
Even if an asynchronous mode is used, synchronous errors are caught.
Mixed Synchronous & Asynchronous
Would Fail:
t.notThrows(done => {
funcWithAsyncCallback(done);
throw new Error('message');
});
t.notThrows(done => {
throw new Error('message');
return funcReturnsPromise();
});
t.throws(fn, [errTestFn], [message])
An assertion that fn
is function that either throws an Error synchronously or asynchronously (via Promise or callback).
Arguments
fn
: function([done]) - code to assert throws- (optional)
done
: function - callback for asynchronous test validateError
: function - function to validate the error
- (optional)
Except for the validateError
argument, this functions as the opposite of t.notThrow()
. That is throws
passes when there is an Error rather passing when there is no Error. For more usage details, look at the notThrows
examples.
Passing errTestFn
allows testing that the Error received is the Error expected.
t.throws(() => {
return funcReturnsPromise();
},
err => {
t.true(err instanceof TypeError);
t.equal(err.message, 'invalid argument');
});
t.count(fn, count, [message])
An asynchronous assertion that fn
eventually executes a callback a precise number of times.
t.count(done1 => {
funcWithCallback(done1);
funcWithCallback(done1);
funcWithCallback(done1);
}, 3);
Running Multiple Suites
There are couple ways to run multiple suites.
-
The easiest (and least flexible) way to just place multiple suites in a single file.
-
A more flexible way is to create one suite per file and create an index file to run them all.
Imagine your
test
directory looks like this:suite1.js suite2.js suite3.js
You could create a file called
index.js
with these contents:'use strict'; require('./suite1'); require('./suite2'); require('./suite3');
Then add the script to your
package.json
file:"scripts": { "test": "node test/index.js" }
Now you can run individual suites or the whole thing from the command line.
node tests/suite1.js npm test
-
The up coming CLI will make it easier to execute multiple suites without needing to maintain an
index.js
.
Custom Reporters
A reporter is a simple constructor function that takes an event emitter as it's only argument. The emitter emit four different events.
function Reporter(emitter) {
emitter.on('start', root => {
console.log('testing started');
});
emitter.on('testEnd', test => {
});
emitter.on('suiteEnd', suite => {
});
emitter.on('end', () => {
});
}
start
Event: Emitted before tests start running.
root => { }
root
: objectsuites
: array - suites that will be run (seesuiteEnd
for suite structure)
testEnd
Event: Emitted after a test has completed.
test => { }
test
: objectname
: string - name of testsuite
: object - suite the test belongs tostatus
: string -pass
,fail
,todo
,skip
, orstop
runTime
: number - run time of test in milliseconds
suiteEnd
Event: Emitted after a suite has completed.
suite => { }
suite
: objectname
: string - name of testerr
: Error|undefined - suite level errors, such as beforeAll and afterAll hook errorstests
: array - all tests in the suite (seetestEnd
for test structure)
end
Event: Emitted when all suites are completed.
Todo
- CLI
- file watcher
- only run certain files (glob matching)
- harmony flag
Considering
- code coverage
- slow tests
- logging - ala Karma
- support CoffeeScript, TypeScript
- support plugins