Pre-train model about efficientnet-l2-noisystudent
Closed this issue · 10 comments
First, thanks for this amazing work!
I have checked the repo you referred to in your paper about the pre-train model: efficientnet-l2-noisystudent. But in this repo, there is only the model but does not include the pre-train model. Therefore, do you use the efficientnet-l2-noisystudent from its official repo (that is, based on the Tensorflow)?
Sorry I might be misunderstanding your question. We use the EfficientNet-L2-NoisyStudent from the pytorch-image-models repo which itself has been ported from the official tensorflow repo. Are you having any trouble running the model?
Sorry I might be misunderstanding your question. We use the EfficientNet-L2-NoisyStudent from the pytorch-image-models repo which itself has been ported from the official tensorflow repo. Are you having any trouble running the model?
Hi @rtaori,
I mean the model of EfficientNet-L2-NoisyStudent from this repo: pytorch-image-models repo is based on the PyTorch, but the weight of the model of EfficientNet-L2-NoisyStudent from its official repo is based on TensorFlow.
So I guess you use the TensorFlow-based EfficientNet-L2-NoisyStudent model to conducts the experiments. Or you have the PyTorch-based EfficientNet-L2-NoisyStudent pre-train weights.
Yes, we use the pytorch version - to be clear, the weights are exactly the same. The weights are ported from the official tensorflow repo into the pytorch model. We use pytorch so it integrates with the rest of our testbed, but if you run the tensorflow model you should get the exact same results (up to some small differences in data-preprocessing if you use tensorflow's dataloaders).
Yes, we use the pytorch version - to be clear, the weights are exactly the same. The weights are ported from the official tensorflow repo into the pytorch model. We use pytorch so it integrates with the rest of our testbed, but if you run the tensorflow model you should get the exact same results (up to some small differences in data-preprocessing if you use tensorflow's dataloaders).
Thanks for your quick reply and again really appreciate this amazing work. EfficientNet-L2-NoisyStudent is the only released pre-train model that is trained on JFT-300M :), so it is very important.
Ah yes, no worries! Let us know if you have any more questions. Closing this issue as it seems resolved.
Hi Rohan:
Do you have some tools for converting TensorFlow weights to Pytorch? I try some tools from GitHub, but it seems like not work.
Hi,
Sorry, I don't have any custom tools I built myself. So far I've just used ports others in the community have created :)
-Rohan
Hi,
Sorry, I don't have any custom tools I built myself. So far I've just used ports others in the community have created :)
-Rohan
So, how can convert Noisy-student pre-train to PyTorch, could you please give some instructions? torch.load() can not load TensorFlow checkpoint.
If you're looking to do the conversion yourself, you could ask https://github.com/rwightman for his script. If you're just looking to use the model in pytorch, you can either use the model from our testbed for from the https://github.com/rwightman/pytorch-image-models repository.
If you're looking to do the conversion yourself, you could ask https://github.com/rwightman for his script. If you're just looking to use the model in pytorch, you can either use the model from our testbed for from the https://github.com/rwightman/pytorch-image-models repository.
Thanks, I find the PyTorch weight of noisy-student is already in this repo.