Resume training
BlakeOne opened this issue · 1 comments
Is there any way to resume training? EG if after testing my trained LORAs in comfy I want to train them for another few hundred steps...
Currently, resuming training is not available as this is a "one shot" finetuning approach.
I may implement it in the coming days, but I would need to test it as the format of the tensor keys are changed (when saving) to work with the community built UIs.
A possible, "hacky" workaround may be to use the CheckpointSave Node in ComfyUI after loading the spatial LoRA, convert to Diffusers format, then load it here for training, but I can't confirm the same would work for the motion modules.
You can manually define the number of steps here to train longer (just set -1 to something high like 10000), and then stop when you think the result is good enough.
Hope that helps!