tristandeleu/pytorch-maml-rl

Question about first_order argument

Opened this issue · 1 comments

hzyjerry commented

Hi,

I am wondering what are the effects of having first-order=False, and when should we use it?

From what I understand of the current implementation, first_order only affects the sample method, and thus is not reflected in MAML training loop (MetaLearner.surrogate_loss, MetaLearner.step). I am wondering if this would make any difference in training.

Would like to confirm with you on this.

Thanks

tristandeleu commented

Hi,
Thanks for pointing that out! You are absolutely right, the first_order parameter has very little impact right now. The idea of first_order was to have a setting similar to the one described in Section 5.2 of the original paper (in the context of supervised learning), where any second order derivatives are ignored for the optimization.

Right now this is not correctly implemented, and I apologize for that. I didn't work with that setting at all, and I only added that as a "bonus". I'll make the according changes as soon as possible.

  • 👍1