matlab_autotest

Tool to add automatic checkpoints to messy code (that can later be turned into unit tests).

This is not meant to be a replacement for a carefully written Testing Framework. Rather, it is a development tool to refactor code that has little or no tests; to make sure that, as you tidy up the code (and write propper tests) it continues to produce the same outputs.

Structure

The repo defines three classes:

TestCheckpoint defines individual test objects (see Usage below)
WorkspaceBackup is a wrapper around save. It allows TestCheckpoint to do workspace dumps from the caller environment, with additional options (e.g. to exclude certain classes).
TestSession is used to store a static (global-like)¹ index of test objects. It allows these to be accessed from anywhere in the code, via the structure TestCheckpoint.session.tests.

Usage

1. Define tests

Test objects include a unique id, and input and output options (WorkspaceBackup objects) that define which variables are to be (re)stored before and after the code to be tested.
Every test object will be pushed to the static index, as TestCheckpoint.session.tests.(id)

function define_test_objects()
% Has to run before instrumented code

    % Object will become available as TestCheckpoint.session.tests.test_dummy
    TestCheckpoint('test_dummy', 'input', {'onlynames', {'x','y'}}, 'output', {'onlynames', 'z'});
end

2. Instrument code

Retrieve a test from TestCheckpoint.session.tests
Use obj.do('input') and obj.do('output') to mark the spots right before and after the code to be tested:

function z = instrumented_function(x, y)

    % Get test object -- requires a previous call to TestCheckpoint('test_dummy',...)
    obj = TestCheckpoint.session.tests.test_dummy;

    % insert right before code to be tested
    obj.do('input');

    % Do stuff
    z = dummy(x, y);

    % insert right after code to be tested.
    obj.do('output');
end

2. Do a `'setup'` run

When the state flag of a test object obj is set to 'setup' (default, if there are no backup files associated with id):

obj.do('input') - Will save all reference workspace inputs, into obj.input.file.
obj.do('output') - Will save all reference workspace outputs, into obj.output.file.

3. Modify the code

While obj.state == 'idle' (once reference files have been stored in a setup run), calls to obj.do will not do anything.

4. Do a `'test'` run

If the state flag of a test object obj is set to test (this must be done explicitly):

obj.do('input') - Will restore all reference workspace inputs, from into obj.input.file.
obj.do('output') - Will save all test workspace outputs, into obj.test_io.file.

Compare the contents of obj.test_io.file and obj.output.file

TODO: This should be done automatically, perhaps generating a test-script with references to the test files

5. (Optional) reset

Use TestCheckpoint.session.reset(true) to delete all test objects and their associated files.

https://uk.mathworks.com/help/matlab/matlab_oop/static-data.html ↩

UoMResearchIT/matlab_autotest