test-all-versions

Run your test suite against all published versions of a given dependency.

Usage

Use the tav command to run the tests:

$ tav [options] [<module> <semver> <command> [args...]]

Example running node test.js against all versions of the mysql module that satisfies the ^2.0.0 semver:

tav mysql ^2.0.0 node test.js

options

--help - Output usage info
--quiet - Don't output stdout from tests unless an error occors
--verbose - Output a lot of information while running
--compat - Output just module version compatibility - no errors
--ci - When running tav together with a .tav.yml file, use this argument to only run the tests on a CI server. This allows you to add tav to your npm test command without spending time running tav tests in development.

.tav.yml

If tav is run without specifying a module, it will instead look for a .tav.yml file in cwd and expect that to contain all its configuration. This is similar to how Travis CI works with .travis.yml.

The following is an example .tav.yml file that runs a subset of tests against all versions of the mysql module that satisfies ^2.0.0 and all versions of the pg module that satisfies *:

mysql:
  versions: ^2.0.0
  commands: tape test/mysql/*.js
pg:
  versions: "*"
  commands:
    - node test/pg1.js
    - node test/pg2.js

Node.js version

You can optionally specify a node key in case you want to limit which verisons of Node.js the tests for a specific package runs under. The value must be a valid semver range:

mysql:
  node: ">=1.0.0"
  versions: ^2.0.0
  commands: node test/mysql.js

Peer Dependencies

If a package or a test needs certain peer dependencies installed in order to be able to run, use the peerDependencies key. The value can be either a single value like shown below, or a list of values just like with the commands key:

graphql-express:
  peerDependencies: graphql@0.9.2
  versions: ^0.6.1
  commands: node test/graphql-express.js

Setup and Teardown

If you need to run a script before or after a command, use the preinstall, pretest and posttest keys:

graphql:
  versions: ^0.7.0
  preinstall: rm -fr node_modules/graphql-express
  commands: node test/graphql.js

Usage:

preinstall: runs before npm install
pretest: runs before each command in the commands list
posttest: runs after each comamnd in the commands list

Multiple test-groups per module

If you need multiple test-groups for the same module, use - to specify an array of test-groups:

mysql:
  - versions: ^1.0.0
    commands: node test/mysql-1x.js
  - versions: ^2.0.0
    commands: node test/mysql-2x.js

Test matrix

If you specify environment variables using the env key, the test commands will be run once per element in the env array. In the following example node test/mysql.js will run twice for each version matching ^2.0.0 - once with MYSQL_HOST=server1.example.net MYSQL_PWD=secret! and once with MYSQL_HOST=server2.example.net.

mysql:
  env:
    - MYSQL_HOST=server1.example.net MYSQL_PWD=secret!
    - MYSQL_HOST=server2.example.net
  versions: ^2.0.0
  commands: node test/mysql.js

If more than one test-case is needed for a given module, the environment variables can shared between them using the following syntax:

mysql:
  env:
    - MYSQL_HOST=server1.example.net MYSQL_PWD=secret!
    - MYSQL_HOST=server2.example.net
  jobs:
    - versions: ^1.0.0
      commands: node test/mysql-1x.js
    - versions: ^2.0.0
      commands: node test/mysql-2x.js

Whitelist tests with environment variables

You can use the enironment variable TAV to limit which module from the .tav.yml file to test:

TAV=mysql

This allows you to create a build-matrix on servers like Travis CI where each module in your .tav.yml file is tests in an individual build. You can also comma separate multiple names if needed:

TAV=mysql,pg

To see an example of this in action, check out the .travis.yml and .tav.yml files under the Elastic APM Node.js Agent module.

Prerelease versions

Set to suppress the default behavior of excluding prerelease tagged versions.

mysql:
  versions: ^2.0.0
  commands: tape test/mysql/*.js
  includePrerelease: true

License

MIT

YanivD/test-all-versions