How to train tieredimagenet？
Closed this issue · 7 comments
@icoz69 Thanks for your excellent work! When I train tieredimagenet, should I change the learning rate, max_epoch, step_size and other parameters? Or should I keep the pre-defined default parameters unchanged?
hello, you can use the default hyper-parameters to train all datasets, but you may change it a little bit to get the best performance, though they are close. My training practice is to only tune the learning rate at the meta-learning stage. For example, if you find the validation acc saturates very quickly, then you can try a small lr.
@icoz69 When I was training your model on tieredimagenet, I encountered some problems. I train as follows. First, train the pretrain model according to the default parameters you give. Then use the default parameters to train the meta stage. We only got 68.30 acc on validation. We adjusted the learning rate lr from 0.0005 to 0.0001 according to the suggestions you gave. The model achieved the highest accuracy rate of 68.25 acc in the first epoch. In order to eliminate the influence of the pretrain stage, we used the pretrain model you gave when training the meta stage, but still only got 68.04acc. Can you give me detailed training strategies so that I can reproduce your results?
hi, when we were training the model on other datasets,
we only adjust step_size and gamma during pre-training, and lr during meta-training.
so, you may try other configurations during pre-training, such as step_size=20, while gamma=0.3. since the number of training images varies for different datasets, the optimal lr schedule may also vary.
@icoz69 Thank you for your reply. If I initialize with the pretrain model you gave, how much should the lr of the meta stage be set on tierdimagenet?
the provided pretrain models are not necessarily the optimal ones. i train these models using the released codes. so, you may try different values. my practice is that if the val acc stop increasing very early, I set a lower lr.
@icoz69 Thanks for your advice. I used your default parameters. During the training of the meta stage, although the val acc was only 68.30, the test acc reached 71 in the final test stage. Your code is very robust, thanks for your excellent work!
you're welcome. good luck to your project!