replicate/replicate-python

Increase the timeout

smyja opened this issue · 2 comments

smyja commented

Some models take up 6 minutes to get started.It would be better to add a timeout parameter to the run function to increase the timeout.

mattt commented

@smyja Sorry, by timeout are you referring to the polling interval?

The replicate.run method creates a prediction and polls for updates until the prediction is complete. The initial call to create a prediction should complete quickly and return a prediction in starting status. And subsequent requests to get the latest status should also complete quickly, and not time out.

smyja commented

@smyja Sorry, by timeout are you referring to the polling interval?

The replicate.run method creates a prediction and polls for updates until the prediction is complete. The initial call to create a prediction should complete quickly and return a prediction in starting status. And subsequent requests to get the latest status should also complete quickly, and not time out.

I was using fastapi and it kept timing out. I resolved 8t by switching to async_run. I'll close the issue now.

  • 👍1