detach
Opened this issue · 1 comments
SmileTAT commented
why use detach() for promptB？Is it the solution for "Parameter at index 1 with name xxx has been marked as ready twice"?
zengyh1900 commented
Hi @SmileTAT
detach() means setting the gradients to zero for the specified parameter. Here, we want to optimize only one prompt, namely PromptA, so we detach PromptB. TBH, we haven't tried not detaching PromptB; they might have a similar performance. You could give it a try.