This is an Extension for the Automatic1111 Webui, which attempts to clean up memory after each generation.
- A few Reddit posts/comments mentioned that CUDA sometimes can cause memory issues/leaks.
This Extension tries to solve that by calling
torch.cuda.empty_cache()after each generation.
- Perform
gc.collect()andtorch.cuda.empty_cache()after every generation - A button to manually trigger the above
- A button to unload then reload the checkpoint
- (This is just the same built-in functions in the Actions section of the Settings tab)
You can optionally go to the System section of Settings tab to enable Unload Checkpoint after Generation.
This makes it so that Checkpoint is only kept in memory during generation, thus achieving the most memory reduction,
but the reloading may take a while depending on your system specs.
[1] Shout out to @kgmkm_mkgm for sharing this Extension with tens of thousands of people
[2] Apparently, this indeed does help in certain situations