proteneer/timemachine

Fixing failing tests that are skipped due to speed

Closed this issue · 6 comments

Going through several tests that we had skipped due to time, noticed that some no longer pass.

  • slow_tests/test_mtm_condensed.py::test_condensed_phase_mtm
assert np.abs(np.average(vacuum_torsions, weights=vacuum_weights)) < 0.1
E       AssertionError: assert 0.11419770372703592 < 0.1
E        +  where 0.11419770372703592 = <ufunc 'absolute'>(0.11419770372703592)
E        +    where <ufunc 'absolute'> = np.abs
E        +    and   0.11419770372703592 = <function average at 0x7fadc499a680>(DeviceArray([-1.35440997, -1.09264612, -0.80219074, ..., -0.29532676,  0.65361984,  1.97714362], dtype=float64), weights=array([2.14505981e-06, 8.92347275e-05, 7.54157375e-21, ..., 1.66304206e-93, 4.84447140e-14, 8.26169455e-06]))
E        +      where <function average at 0x7fadc499a680> = np.average

  • slow_tests/test_vacuum_importance_sampling.py::test_vacuum
assert np.mean((enhanced_torsions_lhs - vanilla_samples_lhs) ** 2) < 5e-2
E       assert nan < 0.05
E        +  where nan = <function mean at 0x7fad8b193a70>(((array([0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 9.64828802e-04, 2.89448641e-03, 2.89448641e-02, 2.21910624e-02, 1.09668874e-01, 3.40584567e-01, 4.00082343e-01, 5.81148548e-01, 7.13008485e-01, 7.50636808e-01, 8.38114619e-01, 8.40687496e-01, 6.85028449e-01, 7.51601637e-01, 5.68927384e-01, 6.57370024e-01, 5.25510088e-01, 5.63460020e-01, 5.63781630e-01, 5.89188788e-01, 5.92726494e-01, 7.81189720e-01,\n       1.00760288e+00, 8.95361128e-01, 7.68646946e-01, 9.33954280e-01, 6.86636497e-01, 2.97810490e-01, 2.02292439e-01, 1.52121341e-01, 2.57287681e-02, 2.92664737e-02, 7.71863042e-03, 6.43219201e-04, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00]) - array([nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan])) ** 2))
E        +    where <function mean at 0x7fad8b193a70> = np.mean

We may want to schedule nightly tests for these sorts of tests to ensure long term coverage

After some digging the issue with the test_vacuum test. The nans come from from this line: https://github.com/proteneer/timemachine/blob/master/slow_tests/test_vacuum_importance_sampling.py#L90

Looks like none of the torsions for the vanilla end up on the left hand side.

Only PR that affected torsions that I can think of is:

https://github.com/proteneer/timemachine/pull/626/files

Only PR that affected torsions that I can think of is:
https://github.com/proteneer/timemachine/pull/626/files

Checking if reverting that PR fixes this test.

Locally, this fails on master at a different assertion:

assert num_batches % num_workers == 0

@maxentile I had seen that and modified it to be 8, but had thought because I have 24 cores it was an atypical number, not sure how many cores @proteneer has. I also tried reverting the commit with it set to 8 cores and there was no change in the test failure. Perhaps this still passes on @proteneer machine and it never passed elsewhere?

Hm good call - lemme double check.