what the dropout should be set when we predict or test?
Closed this issue · 2 comments
hitwangshuai commented
Hello, I have a question, what the dropout should be set when we predict or test? thank you
zhangzhenyu13 commented
Do this when you are training your model.
KL loss serves as penalty so the model parameters will not change significantly each step, thus resulting in an better optima.
The KL loss in such a way works like a regularizer but is much more efficient than other regs.
When you eval/test, dropout is turned off and predictions are same, --> no more need to apply R-Drop.
dropreg commented
Hello, I have a question, what the dropout should be set when we predict or test? thank you
As zhangzhenyu13 said that，It don't have to do anything for the inference (predict or test) stage.