sbi-dev/sbi

Slow L-C2ST test fails locally

Closed this issue · 2 comments

Describe the bug
L-C2ST (slow) tests fail locally.

FAILED tests/lc2st_test.py::test_lc2st_true_positiv_rate[LC2ST_NF] - AssertionError: LC2ST p-values too small, test should be accepted         at least 95.0% of the time, but was accepted         only 45.0% of the time.

To Reproduce

pytest tests/lc2st_test.py::test_lc2st_true_positiv_rate[LC2ST_NF]

On the current main, Python 3.11.7. (might dependents on python version)

This might be due to insufficient seeding or to strict tolerance for test.

@JuliaLinhart can you have a look at this please?

reason: for this test, the density estimator has to be off, but it was using too many simulations and became too good.

fixed with #1209