noib3/nvim-oxi

Test macro doesn't seem to pick up on failures

Closed this issue · 6 comments

turboladen commented

I'm not sure if I'm just missing something or not, but it seems like the #[nvim_oxi::test] macro isn't doing anything for me.

pub fn is_pum_visible() -> nvim_oxi::Result<bool> {
    let result: u32 = nvim_oxi::api::call_function("pumvisible", nvim_oxi::Array::new())?;

    if result == 0 {
        Ok(false)
    } else {
        Ok(true)
    }
}

#[cfg(test)]
mod tests {
    use super::*;

    use nvim_oxi::{self as oxi};

    #[oxi::test]
    fn is_pum_visible_test() {
        eprint!("hi");
        assert!(is_pum_visible().unwrap());
        panic!("hi");
    }
}

I then run cargo build and cargo test and get:

running 1 test
test tests::is_pum_visible_test ... ok

My test, of course, initially didn't have the eprintln!() or panic!() calls--I just added those to get the test to fail, which it didn't.

  • nvim-oxi: 0.4.1
  • NVIM v0.9.5
  • Build type: Release
  • LuaJIT 2.1.1703358377
  • Rust 1.75.0
  • MacOS 14.2.1
noib3 commented

I can reproduce this if I run it as a unit test.

Is that file your lib.rs? Could you also post the Cargo.toml? I'll try debugging this tomorrow.

turboladen commented
turboladen commented

Ok, so I'm working in a multi-crate workspace; dunno if that changes anything. Here's the workspace Cargo.toml:

[workspace]
members = ["crates/*"]
resolver = "2"

[workspace.dependencies]
log = "0.4"
nvim-oxi = { version = "0.4.1", features = ["neovim-0-9"] }

And the Cargo.toml from the crate I was adding tests for:

[package]
name = "nvim-sous_chef-complete_fn"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = ["cdylib", "rlib"]

[dependencies]
log.workspace = true
nvim-oxi.workspace = true

[dev-dependencies]
nvim-oxi = { workspace = true, features = ["test"] }

FWIW, if I don't scope my test module to #[cfg(test)], tests works as expected (assuming, of course, I enable oxi's test feature in the [dependencies] section), so in the meantime, I've been getting by just fine like that (not ideal in the long run, of course).

noib3 commented

The #[nvim_oxi::test] macro spawns a Neovim process in which the dynamic library built by cargo build is loaded. The problem is that code fenced by a #[cfg(test)] is not included in the build, so when it's time to run the tests we can't even find the entrypoint function.

You don't get an error in this case because we look at the exit code of that Neovim process, not its stderr. But if you manually run

$ nvim -u NONE --headless -c "set rtp+={path-to-your-crate}/target/debug/oxi-test" -c "lua require('__is_pum_visible_test')" +quit

you should see something like:

Error detected while processing command line:
E5108: Error executing lua vim/_init_packages.lua:0: dlsym(0x7ff9046a8e10, luaopen___is_pum_visible_test): symbol not found
stack traceback:
	[C]: in function 'error'
	vim/_init_packages.lua: in function <vim/_init_packages.lua:0>
	[C]: in function 'require'
	[string ":lua"]:1: in main chunk⏎

This also explains why:

if I don't scope my test module to #[cfg(test)], tests works as expected

The best workaround I can currently think of is to add a tests feature to your crate, and modify the cfg to

#[cfg(any(test, feature = "tests"))]
mod tests { .. }

then you can build the crate with cargo b --features=tests, and run the tests with cargo t --features=tests.

Definitely not ideal, but I'm not sure if there's a better solution.

turboladen commented

Ah yeah, that all makes sense--thanks for looking into it. And that's certainly a reasonable workaround--one that hadn't occurred to me--thanks!

noib3 commented

I'll close this for now but feel free to re-open it if you're still experiencing similar issues.

  • 👍1