automated assert retries

[codan84]

What am I trying to do

I am trying to run smoke tests on an async, event-based, system.
Since these tests run on an actual, deployed code, sometimes it will take few hundred milliseconds before events are processed and output of these can be queried (ie cold starts on AWS Lambda).
I would like tests to retry assertions X times on an X [ms] interval if they fail.

What I did

Here is a function I wrote to handle this:

export const retryAssertAsync = async (t, assertFn, retriesLeft = 10, interval = 500) => {
  const currentTry = await t.try(async (tt) => {
    await assertFn(tt)
  })
  if (!currentTry.passed && retriesLeft) {
    console.log(`>> Retries left: ${retriesLeft - 1}`)
    currentTry.discard()
    await new Promise(resolve => setTimeout(resolve, interval))
    return retryAssertAsync(t, assertFn, retriesLeft - 1, interval)
  }
  return currentTry.commit()
}

And here is how I use it (one example test):

test('Can add <<data>>', async t => {
  const input = randomInput(TEST_ID)
  await sendMessage(input, 'TEST')

  await retryAssertAsync(t, async (t) => {
    const processedData = await getProcessedDataForTest(TEST_ID)
  
    t.is(processedData.length, 1)
    t.is(processedData[0].foo, input.foo)
  })
})

Suggested change

As you can see I am making a use of .try.
However, I think it'd be great if maybe there was something vaguely similar built-in? For example:

test('Can add <<data>>', async t => {
  const input = randomInput(TEST_ID)
  await sendMessage(input, 'TEST')

  t.retry((tt) => {
    const processedData = await getProcessedDataForTest(TEST_ID)
  
    tt.is(processedData.length, 1)
    tt.is(processedData[0].foo, input.foo)
  }, 5 /**retries**/, 500 /**ms interval**/)
})

[novemberborn]

t.try() is meant to be the low-level primitive for this kind of functionality. Other features can be build on top but need to start in external packages before we consider including it in AVA. await retry(t, async tt => {}) or something like that.