KohakuBlueleaf/LyCORIS

alpha for BOFT or diag_oft

Closed this issue · 3 comments

rockerBOO commented

Is alpha used for BOFT/diag_oft? I'm seeing it being set and stored but not seeing it being used in the code for scaling.

There is a multiplier, though? Should we be using the multiplier instead? But that isn't saved after training?

Thanks

KohakuBlueleaf commented

multiplier is the weight on whole net (like the weight in sd-webui)

for BOFT/OFT I think the original paper doesn't have any alpha-like scaling.

rockerBOO commented

Would it make sense to remove storing the alpha for BOFT/OFT modules? Or would it set for consistency with other modules? I have had people ask me what to set it as and having it stored sends a mixed signal at least.

KohakuBlueleaf commented

I checked
the alpha is for storing constraint.
so it is not used becuz in LyCORIS we don't call it alpha.
we just need to store it into the state_dict