ExplainableML/czsl

Code for image retrieval

Closed this issue · 2 comments

Pulkit-Khandelwal commented

Hi there,
I see that the test code predicts the attr and obj for the given image which is one of the task, But, there is a second task of image retrieval. Given an object and attribute category, find the corresponding retrieval image. I was wondering where can I find the code for image retrieval?
Many thanks!!

Pulkit-Khandelwal commented

@mancinimassimiliano Any update on this? thank you so much!!

mancinimassimiliano commented

Hi @Pulkit-Khandelwal and I deeply apologize for this huge delay. I wanted to answer with a simple script but I did not have time to produce it yet.

My short answer is:

  1. use the same return_images flag mentioned here to get the path of the images in the data list.
  2. Loop over the images (e.g. val/test ones) and, for each image store both its path, its label, and the score for your query composition. For the latter, you should know its index in the output of the model, which is easy via e.g. testset.all_pair2idx[pair].
  3. Check which image in your stored list got the highest score. You may then just plot it as qualitative or, for quantitative results, check if the label of the image is the same as your query pair.

Again, sorry for this delay. I guess the answer is not that useful anymore, but if it is and you have questions, please do not hesitate to ask.