replicate/replicate-javascript

`wait` param for `replicate.run` doesn't work

Closed this issue · 1 comments

zhuk-aa commented

This morning meta/meta-llama-3-70b-instruct was failing to run. I tried adding wait param to make it fail if no response comes in 10s but it hasn't changed anything.
const output = await replicate.run("meta/meta-llama-3-70b-instruct", { input, wait: 10 });

Now the model is up again, but adding wait doesn't change anything.

Proposal:

  • make sure there is a way to define after what time the call would timeout
  • in case wait isn't meant for this - update docs for wait. At the moment it's not clear to me what ti does

Thank you ☀️

aron commented

Ah, I'll look into fixing the docs. The wait is how long the client will keep open a connection to the server before falling back to polling for a response.

If you want to abort the request after a timeout expires you can use an AbortController and the signal option much like a fetch() request.

const controller = new AbortController()
setTimeout(controller.abort, 10 * 60 * 1000)
const output = await replicate.run("meta/meta-llama-3-70b-instruct", { input, signal: controller.signal })
  • 👍1