kienduynguyen/BoxeR

Instance Attention

Opened this issue · 3 comments

Git-Hoon commented

Hello, I would like to know if the instance attention could add into normal transformers such as ViT? And could you please upload the saved models if possible (to 2647625722@qq.com) and thanks a lot for that!!!

kienduynguyen commented

I think it is possible to try adding them into ViT but it would take some adaptations :D. Which kind of saved models do you want?

Git-Hoon commented

Thanks and I finally let this model run on my device. I try to add it into normal transformers such as ViT, but in <box_attention_func.py>, it says <Cannot find reference 'ops' in 'init.py'> and so I don't know where the functions <ops.box_attn_forward()> and <ops.instance_attn_forward()> head for (head for c++ files?). My python version is 3.8.

kienduynguyen commented

you can find the registration of these functions here: https://github.com/kienduynguyen/BoxeR/blob/main/e2edet/module/ops/src/vision.cpp

My recommendation is to implement ViT code inside BoxeR repo or you can copy the whole e2edet/module/ops folder to your repo but remember to rebuild the cuda code.