rust-lang/rust

Add ability to ignore tests at runtime.

XAMPPRocky opened this issue · 4 comments

XAMPPRocky commented

For medium–large scale projects, or projects that have a lot of integrations with third party services. It would nice to be able to ignore tests based on the environment the tests are being run in. One example would be to run tests if an environment variable is present.

You can workaround this with something like the following, however the output of the test is ok and not ignored, misleading the user that the test was successful.

#[test]
fn foo() {
    if env::var("FOO").is_none() { return; }
    let foo = env::var("FOO").unwrap();
}
Mark-Simulacrum commented

I would personally rather leave this to custom test frameworks; I think libtest should try to be minimal.

It would also be good to spec this out a bit more. I imagine maybe a proc macro would be enough (i.e., inserting the env) and allowing #[test] to return something like enum TestResult { Passed, Failed, Ignored }.

ArniDagur commented

This is a pain point which is encountered in uutils/coreutils#1041 (comment)

yanganto commented

This crate helps you ignore test case when the environment variable is absent.
https://crates.io/crates/test-with

https://github.com/yanganto/test-with/blob/6ac0336a5f47558a467bd245427a540fdad6f46e/examples/env.rs#L3-L13

#[cfg(test)]
mod tests {
    #[test_with::env(PWD)]
    fn test_works() {
        assert!(true);
    }
    #[test_with::env(NOTHING)]
    fn test_ignored() {
        panic!("should be ignored")
    }
}