Lightning-Universe/stable-diffusion-deploy

Issues in framework affecting autoscaling

rohitgr7 opened this issue · 2 comments

  1. We need to register new works when traffic increases, but every time we spawn a new work, it takes a lot of time to provision, and most of the time is spent on installing the requirements even though it's the same in all the works. It should cache the requirements.
  2. Once work is stopped, it can't be restarted, so we used a workaround to use UUID once the app is running for too long, it creates a mess in the UI.

Also, not so pretty hack of using setattr for defining Works

yeah , there's a PR already on the platform side to resolve List and Dict structures.