facebookresearch/moco-v3

Tensorflow version

wildphoton opened this issue · 7 comments

wildphoton commented

Thank you for open source the Pytorch implementation. I wonder if the original tensorflow implementation have been released for the purpose of training on TPUs.

endernewton commented

We have not released and there is no plan to release the code. This implementation is supposed to be a faithful re-implementation of the TPU code, with details such as initialization (though still subject to framework differences).

endernewton commented

The mmf team (https://github.com/facebookresearch/mmf), specifically @ronghanghu have a good implementation of MoCo v3 using PyTorch XLA, which could be a good starting point on TPUs. Maybe good to check about their plan for releasing it.

wildphoton commented

Thanks!

ronghanghu commented

@wildphoton I can share with you a copy of our TPU implementation w/ PyTorch XLA if you are interested.

wildphoton commented

@wildphoton I can share with you a copy of our TPU implementation w/ PyTorch XLA if you are interested.

Sure. It would be great to have access to it!

MukundVarmaT commented

hi @ronghanghu @wildphoton,
Can you please share the TPU implementation w/ torch xla? If it's already hosted on a different repo, could you please point me to the same?

Thanks in advance.