UnitTesting-example

This is an simple example to use UnitTesting to test a Sublime Text package on a local machine and via continuous integration services such as travis-ci and appveyor.

For testing syntax_test files, go directly to testing syntax_test files.

Preparation

Before testing anything, you have to install UnitTesting via Package Control.
Your package! In our case, it is helloworld.py
TestCases should be placed in test*.py under the directory tests (configurable, see below). They are loaded by a modified TestLoader, see TestLoader.discover for more details. (Sublime Text 2 developers should read this for unittest documentation).
Take a look of the file test.py under the tests directory for a minimal example.

Running Tests

UnitTesting can be triggered via the command palette command UnitTesting. Enter the package name in the input panel and hit enter, a console should pop up and the tests should be running. To run only tests in particular files, enter <Package name>:<filename>. <filename> should be a unix shell wildcard to match the file names, <Package name>:test*.py is used in default.

You could run the command UnitTesting: Test Current Project to run the current project. The current project will be first reloaded (see next section) by UnitTesting and then the tests will be run.

Reloading package (Sublime Text 3 only)

Sublime Text package developers may find themselves have to close and re-open the software multiple times when developing a package. The command UnitTesting: Reload Current Project would reload the current project so developers do not have to restart Sublime Text.

Test Coverage (Sublime Text 3 only)

Furthermore, it is also possible to check test coverage via coverage. The corresponding command is UnitTesting: Test Current Project with Coverage.

Travis and Appveyor

If the tests can be run locally, let's put them to travis-ci and let travis-ci takes care of them. First, you have to copy a important file: .travis.yml (caution: with a beginning dot) to your repo. Then change the env variable PACKAGE in .travis.yml to the name of your package. Don't forget to login travis-ci and enable travis-ci for your repo. Finally, push to github and wait..

To enable Appveyor for windows platform tests, copy the file appveyor.yml to your repo, change the PACKAGE variable in appveyor.yml. The last but not least, login appveyor to add your repo as a project.

Coverage and Coveralls.io support on Travis (For Sublime Text 3)

To generate coverage report for coveralls.io, you just have to specific three things in .travis.yml

run the test with the --coverage flag
```
sh travis.sh run_tests --coverage
```
install python-coveralls
run coveralls after success

Check .travis.yml for details. The file .coveragerc is used to control the coverage configuations. If it is missing, UnitTesting will ignore the tests directory.

Installing Package Control and Dependencies

If your package uses Package Control dependencies, you may want to install Package Control by umcommenting the line of install_package_control in .travis.yml or appveyor.yml.

Testing syntax_test files

To enable testing of the syntax_test files, please copy the .travis.yml or appveyor.yml, and use the run_syntax_tests in those files. Check syntax branch for an example.

Options

Use a different test directory

The default test directory is "tests". To change the test directory, add a file unittesting.json to your repo with the corresponding directory name, eg unittest:

{
    "tests_dir" : "unittest"
}

Redirect test result to a file

The test result could be redirected to a file by specifying the output variable in unittesting.json.

{
    "output" : "foo.txt"
}

Deferred testing

Tests can also be written using the Deferrable testcase, such that you are able to run sublime commands from your test cases and yield control to sublime text runtime and continue the execution later. Would be useful to test asynchronous codes.

A example would be found in deferred branch.

To activate deferred testing on travis and appveyor. Add the file unittesting.json to your repo with the following:

{
    "deferred": true,
}

Async testing (Sublime Text 3 only)

In default, the tests are running in the main thread and can block the graphic inference. Asychronized testing could be used if you need the interface to respond.

Async tests are usually slower than the sync tests because the interface takes time to respond but it is useful when there are blocking codes in the tests. A example would be found in async branch.

However, it is known that async test does not work very well with coverage. In general, it is recommended to use deferred testing over async testing since there is no need to worry about race condition.

To activate async testing on travis and appveyor. Add the file unittesting.json to your repo with the following:

{
    "async": true,
}

Note:

async is forced to be false on Sublime Text 2
if async is true, deferred is forced to be false (relaxation of this is in progress)

Vagrant (Outdated)

Debugging in travis-ci could be difficult. To mock the travis-ci environment in your computer, you can use vagrant. For most users, this section could be ignored.

# clone the example to somewhere
git clone https://github.com/randy3k/UnitTesting-example
cd UnitTesting-example
# you can also launch `st2` config
vagrant up st3
# enter ssh
vagrant ssh st3
# run tests
sh vagrant.sh run_tests

truekonrads/UnitTesting-example