spec

What is it?

spec is a Python testing tool that turns this:

into this:

Specifically, spec provides:

Colorized, specification style output
Colorized tracebacks and summary
Optional CLI tool which enables useful non-default options and implements relaxed test discovery for less test_annoying.py:TestBoilerplate.test_code and more readable.py:Classes.and_methods.

Details

spec is a BDD-esque nose plugin designed to provide "specification" style test output (similar to Java's TestDox or Ruby's RSpec). Spec-style output provides a more structured view of what your tests assert, compared to nose/unittest's default "flat" mode of operation.

For example, this nose-style test module:

class TestShape(object):
    def test_has_sides(self):
        pass

    def test_can_calculate_its_perimeter(self):
        pass

class TestSquare(object):
    def test_is_a_shape(self):
        pass

    def test_has_four_sides(self):
        pass

    def test_has_sides_of_equal_length(self):
        pass

normally tests like so, in a single flat list:

TestShape.test_has_sides ... ok
TestShape.test_can_calculate_its_perimeter ... ok
TestSquare.test_has_four_sides ... ok
TestSquare.test_has_sides_of_equal_length ... ok
TestSquare.test_is_a_shape ... ok

With spec enabled (--with-spec), the tests are visually grouped by class, and the member names are tweaked to read more like regular English:

Shape
- has sides
- can calculate its perimeter

Square
- has four sides
- has sides of equal length
- is a shape

In other words:

Class-based tests are arranged with the class name as the subject, and the methods as the specifications;
Any module-level tests are arranged with the module name as the subject;
All objects' docstrings are used as their descriptions, if found. Otherwise:
- CamelCaseNames (typically classes) have any leading/trailing Test stripped, as well as any trailing underscore;
- CamelCaseNames also get turned into sentences if necessary, so e.g. CamelCaseNames becomes Camel case names;
- underscored_names have any leading/trailing test (with its attached underscore) stripped;
- underscored_names have underscores turned into spaces;

Test runner

spec also ships with a same-name command-line tool which may be used as a more liberal nosetests. In addition to toggling a number of useful default options (such as nose's builtin --detailed-errors) spec-the-program will honor any and all public objects defined within your project's tests directory, meaning any file, function or class whose name does not begin with an underscore ('_') and which is defined locally.

For example, given the following code inside tests/feature_name.py:

from external_module import a_function, AClass

def _helper_function(args):
    return a_function(args)

class _Parent(object):
    def this_will_not_get_tested():
        pass

class Feature(_Parent):
    def should_have_some_attribute(self):
        _helper_function(AClass)

    def does_something_awesome(self):
        self._helper_method()

    def _helper_method(self):
        pass

def something_tested_by_itself_outside_a_class():
    pass

only the following items will be picked up as test cases:

Feature.should_have_some_attribute
Feature.does_something_awesome
something_tested_by_itself_outside_a_class

The imported function and class, the underscored functions/methods, and the methods inherited from a parent class, are all ignored.

Usage tips

Following from spec-the-tool's discovery algorithm, and spec-the-plugin's name transformation, we suggest the following for both readable code and test output:

Store tests in tests/, with whatever file-by-file organization you like best;
Within files, import the classes under test normally, e.g. from mymodule import MyClass;
Name the test classes identically, but with a trailing underscore to avoid name collisions, e.g. class MyClass_(object): [...]
Name their methods like English sentences, e.g. def has_attribute_X(self): [...].

For example:

from mypackage import MyClass, MyOtherClass

class MyClass_(object):
    def has_attribute_A(self):
        pass

class MyOtherClass_(object):
    def also_has_attribute_A(self):
        pass

    def has_attribute_B(self):
        pass

tests as:

MyClass
- has attribute A

MyOtherClass
- also has attribute A
- has attribute B

Activation / command-line use

After installation via setup.py, pip or what have you, nosetests will expose these new additional options/flags:

--with-spec: enables the plugin and prints out your tests in specification format. Also automatically sets --verbose (i.e. the spec output is a verbose format.)
--no-spec-color: disables color output. Normally, successes are green, failures/errors are red, and skipped tests are yellow.
--spec-doctests: enables (experimental) support for doctests.

Why would I want to use it?

Specification-style output can make large test suites easier to read, and like any other BDD tool, it's more about framing the way we think about (and view) our tests, and less about introducing new technical methods for writing them.

Where did it come from?

spec is heavily based on the spec plugin for Titus Brown's pinocchio set of Nose extensions. Said plugin was originally written by Michal Kwiatkowski. Both pinocchio and its spec plugin are copyright © 2007 in the above two gentlemen's names, respectively.

This version of the plugin was created and distributed by Jeff Forcier, © 2011. It tweaks the original source to be Python 2.7 compatible, based on similar changes. It also fixes a handful of bugs such as broken SkipTest compatibility under Nose 1.x, and then adds some additional functionality on top (most notably the spec command-line tool.)

What's the license?

Because this is heavily derivative of pinocchio, spec is licensed the same way -- under the MIT license.

dmr/spec