kevin-ssy/CLIP_as_RNN

Request for clip_wrapper.py Compatible with open-clip for CAM Generation

Opened this issue · 0 comments

First of all, thank you for your excellent research.

I am interested in applying your approach using a CLIP model trained in a different domain.

I would like to know how to obtain CAM results from a pretrained CLIP using open-clip (https://github.com/mlfoundations/open_clip) instead of OpenAI's CLIP.

I attempted to generate CAMs, but due to the different model structure, I am unsure if I have implemented it as you intended. (The results were not as good as expected, so I am reaching out for help.)

Could you provide a clip_wrapper.py that is compatible with open_clip? (I tried it myself, but I am keen to understand your precise implementation.)

I have attached the network in a txt file.
open-clip.txt
open_ai_clip.txt