VoltaML/voltaML-fast-stable-diffusion

[Bug]: Adding new jobs too quickly (or during loading) breaks the job queue

aaronsantiago opened this issue · 1 comments

Describe the bug

Error log

I'm not sure if this is specific to controlnet, but when adding jobs to the queue too quickly the job queue breaks down and volta refuses to accept any more requests. This window where requests break the job queue is very wide while controlnet is loading specifically, and is the easiest way to trigger the bug.

Reproduction

  1. Load a model
  2. Do a controlnet request
  3. Immediately do a second controlnet request (before controlnet is finished loading)

or

  1. Send two controlnet requests within about .3 seconds of each other

Expected behavior

Jobs are added to the queue and are processed when Volta is ready

Installation Method

Local

Branch

Experimental

System Info

WSL, Python 3.9, latest experimental

Logs

No response

Additional context

No response

Validations

  • Read the docs.
  • Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
  • I am writing the issue in English.

Support for concurrent jobs was removed in 3df1765