delvtech/hyperdrive-rs

Remaining Flaky Tests

Closed this issue · 4 comments

jrhea commented

We are ignoring failures with these tests currently. This needs to be addressed ASAP

  • long::targeted::tests::test_calculate_targeted_long_with_budget
  • short::max::tests::fuzz_calculate_absolute_max_short_execute
  • short::max::tests::fuzz_calculate_max_short_no_budget
dpaiton commented

#95 resolves the remaining OP flaky tests, but unfortunately introduces a new one:

  • long::max::fuzz_calculate_max_long

I'm investigating that now

dpaiton commented

another one:

failures:

---- short::max::tests::fuzz_calculate_max_short stdout ----
thread 'short::max::tests::fuzz_calculate_max_short' panicked at crates/hyperdrive-math/src/short/max.rs:723:17:
expected (base=34326359.285477447501009068) < (budget=35401424.462665845134388594) * 0.001000000000000000 = 35401.424462665845134388


failures:
    short::max::tests::fuzz_calculate_max_short
dpaiton commented

#95 resolves the remaining OP flaky tests, but unfortunately introduces a new one:

  • long::max::fuzz_calculate_max_long

I'm investigating that now

resolved by #101

dpaiton commented

another one:

failures:

---- short::max::tests::fuzz_calculate_max_short stdout ----
thread 'short::max::tests::fuzz_calculate_max_short' panicked at crates/hyperdrive-math/src/short/max.rs:723:17:
expected (base=34326359.285477447501009068) < (budget=35401424.462665845134388594) * 0.001000000000000000 = 35401.424462665845134388


failures:
    short::max::tests::fuzz_calculate_max_short

I ran the fuzz test 4 * 1000 times without failure against the branch for this PR delvtech/hyperdrive#1028

I'm going to close the issue; we can re-open one later if we find new flaky tests after v1.0.7 hits with #1028 merged