lllyasviel/ControlNet-v1-1-nightly

How to finetune ?

scarbain opened this issue · 2 comments

scarbain commented

Hi everyone, thank you @lllyasviel for this wonderful tool you made !

I've been trying to finetune one of the latest V1.1 model : openpose.

I managed to modify the tutorial_train.py and tutorial_dataset.py from the original repo to make the training start but it doesn't seem to be following the openpose at all in the current early stages of training (< 10k steps).

I've tried to hack the tool_add_control and tool_transfer_control scripts to create a model with the latest of V1.1 ControlNet Openpose weights that I could feed to tutorial_train but it seems it's restarting from scratch anyway.

Can you provide some guidelines or feedback on how to finetune an existing model please ?

Thanks

lllyasviel commented

Thanks for the issue. see also the main repo for training doc.

Lanhaoran commented

Hey, I am also considering about this issue. Did you find some efficient method to fine-tune the model? Thanks!