How to pick sigma?
ex3ndr opened this issue · 3 comments
ex3ndr commented
Hey everyone, i am tryin to figure out what values of σ aka sigma is meant to be used during training? There are no mentioning of a specific value in papers for some reason.
zvorinji commented
The paper does mention 0.00001 as a starting point but unclear if that’s actually what they used. That said if testing out whether a different number could make the model converge faster, I’d go up (not down) and by orders of magnitude each test, and wouldn’t go above 1. So basically test 0.0001, if better than default, go test 0.001, and so on.
ex3ndr commented
Where did you get this number? I don't see it in the paper