Resetting hidden state during refinement
qzed opened this issue · 2 comments
qzed commented
It looks looks you're resetting the hidden-state during each RAFT-based refinement step. Is this done on purpose?
Lines 317 to 322 in 0dfa361
haofeixu commented
No, just based on intuitions :)
qzed commented
Thanks! Feel free to close this issue (although I think it could be nice to have a comparison whether this leads to some improvement).