Re-implemented in Pytorch
Newbeeer opened this issue · 2 comments
Hi, I try to re-implemented your method in Pytorch. But I find it can't not reach the accuracy reported in your original paper. I think the problems lie in the restart big learning rate(0.5) when using the pre-trained model, cause the accuracy drops sharply.
It may be a problem that re-trains the model with a large learning rate in the beginning. But according to our experience, it can be corrected later along with the training. In fact, if you check the Figure 3, 4, 5 in our paper, you will find the performance drop at the beginning. That is what you say. However, the results increase finally.
In your case, have you achieved the reported performance for the baselines Forward and S-adaptation? If it is not, it may be the implementation issue for all these models. If it is, you can run our tensorflow implementation in your server and validate the results.
Thanks. I will take your suggestions.