Concept Label

[detectiveli]

Dear contributor,

I am a little confused about how the concept label is generated, and it seems special for each image.

And if the Concept Label is special for each image, during the test process, each image should connect with all the sentence. but during the test process, it seems that the image features and caption features are calculated by cosine similarity where they should be independent with each other. This really confuses me a lot.

Looking forward to your reply.

[BruceW91]

Hi, we recently notice that we made a problematic operation in our implementation to share the concept labels of captions of the same image at reference stage, and we lately develop a concept complement strategy to deal with it. We will update the repo and code in the near future. If there are more questions, you can also contact me by e-mail (haoranwang@tju.edu.cn).