Can we use pertained ijpea for transfer learning object detection task?

[satishjasthi]

First of all thank you for sharing such an amazing project 🙂. Can we use the embeddings generated by context encoder of this pertained model for a task like object detection on custom dataset? If yes do you have any suggestions on which type of head architecture(for bbox and class label prediction) can be used with ijepa as backbone for object detection task?

Thank you

[fenaux]

May be you can find the answer in this paper https://arxiv.org/pdf/2111.06377.pdf
Just keep the encoder as a backbone and create a head for object detection. Case of R-CNN is described in this paper.
In annex A.4 the way to access a FPN like structure is also described