cargo-loom

cargo-loom is a Cargo subcommand to automate running Loom testing workflows.

What Does it Do?

Loom is a testing tool for concurrent Rust code. It runs a test many times, permuting the possible concurrent executions of that test under the C11 memory model.

Because Loom is an exhaustive model checker, the same test may be re-run a very large number of times in order to explore every unique path through the program under test allowed by the model. Although Loom uses state reduction techniques to avoid combinatorial explosion, a non-trivial Loom test may run for 100,000s of iterations before a permutation that results in a failure is detected.

Loom's deterministic execution allows the specific chain of events that caused a test to fail to be isolated and stored in a checkpoint file, so that the failing execution can be re-run without having to explore all the paths under which the model succeeds. Once a failing execution has been isolated, logging and location tracking can be enabled to aid in debugging the failing test. However, this test debugging workflow currently requires a number of manual steps.

This is where cargo-loom comes in. This crate provides a cargo subcommand that automates parts of this workflow. Invoking cargo-loom performs the following actions:

Building the test suite with RUSTFLAGS="--cfg loom" enabled
Running the test suite (with support for cargo test-style filtering) to discover failing tests
Rerunning failing tests to generate a checkpoint file for each failure case
Finally, re-running those failing tests a final time with logging and location tracking enabled, so that the failure can be debugged

Checkpoint files are stored according to the hash of the build artifact for the test suite, so when the code under test has not changed, the checkpointed execution may be reused in future runs to display different outputs or change execution parameters.

Quickstart

To install cargo-loom, run:

RUSTFLAGS="--cfg tokio_unstable" cargo install cargo-loom

Once cargo-loom is installed, run

cargo loom

in a Cargo workspace that contains Loom tests, to run those tests using cargo-loom.

Command-Line Interface

cargo loom supports most of the same command-line options as cargo test, including test name filtering and passing additional arguments to the test binary. For example, to run only tests defined in my_loom_tests.rs with names containing interesting_model, run:

cargo loom --test my_loom_tests interesting_model

Additional arguments can be passed to the test binary using --, similarly to cargo test. For example, to pass the --nocapture argument to disable libtest output capturing, run:

cargo loom -- --nocapture --test-threads 1

The cargo loom CLI can also be used to configure Loom's execution parameters. All of the supported environment variables are passed through to the Loom execution. Additionally, they may also be set using command-line arguments. For example, to limit the maximum duration that a Loom model will run for and the number of thread switches per permutation, run:

cargo loom --max-duration-secs 120 --max-branches 1000

For a complete list of supported command-line arguments, run:

cargo loom --help

License

This project is licensed under the MIT license.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in cargo-loom by you, shall be licensed as MIT, without any additional terms or conditions.