donnot finish
Xinyao Nie, Hong Lu, Zijian Wang, Jingyuan Liu, Zehua Guo
Fudan University
This repository is the code for Weakly Supervised Image Retrieval via Coarse-scale Feature Fusion and Multi-level Attention Blocks in ICMR 2019. If you cannot open the link, you can just download the paper which is uploaded in pdf format.
Python 3, PyTorch >= 0.4.0, and make sure you have installed TensorboardX:
pip install tensorboardX
1. Prepare the Dataset
Our work utilized four datasets: In-shop Clothes Retrieval, CUB-200-2011, Stanford Online Products and Cars-196. In this repo, we just focus on CUB-200-2011 dataset. You can use "get_cub_train_file.py" to generate "train.txt" and "test.txt" OR download "train.txt" and "test.txt" directly.
2. Repo Structure
ABIR
|———— data/ # store CUB-200-2011 dataset
|———— cub/
|———— images/
|———— images.txt # you can use image.txt, image_class_labels.txt
|———— image_class_labels.txt # and train_test_split.txt to
|———— train_test_split.txt # generate train.txt and test.txt
|———— train.txt
|———— test.txt
|———— code/
|———— models/
|———— VGG16_V5.py
|———— __init__.py
|———— config.py
|———— Model2Feature.py
|———— train.py
|———— test.py
|———— trainer.py
3. Train the Model
Run "train.py".
We conduct the experiments on all commonly adopted image retrieval task datasets and utilize Recall@K metric for evaluation.
The following table shows the results on the In-Shop Clothes Retrieval dataset. Best results are marked in bold.
| R@ | 1 | 10 | 20 | 30 | 40 | 50 |
|---|---|---|---|---|---|---|
| FashionNet+Joints | 41.0 | 64.0 | 68.0 | 71.0 | 73.0 | 73.5 |
| FashionNet+Poselets | 42.0 | 65.0 | 70.0 | 72.0 | 72.0 | 75.0 |
| FashionNet | 53.0 | 73.0 | 76.0 | 77.0 | 79.0 | 80.0 |
| HDC | 62.1 | 84.9 | 89.0 | 91.2 | 92.3 | 93.1 |
| HTL | 80.9 | 94.3 | 95.8 | 97.2 | 97.4 | 97.8 |
| A-BIER | 83.1 | 95.1 | 96.9 | 97.5 | 97.8 | 98.0 |
| ABE-8 | 87.3 | 96.7 | 97.9 | 98.2 | 98.5 | 98.7 |
| ------------- | ------------ | ------------ | ----------- | ----------- | ----------- | ----------- |
| Our Baseline | 85.4 | 96.1 | 97.3 | 97.8 | 98.1 | 98.3 |
| ABIR w/o SE-block | 88.1 | 96.9 | 97.6 | 98.1 | 98.3 | 98.5 |
| ABIR with SE-block | 89.0 | 97.1 | 98.0 | 98.4 | 98.6 | 98.8 |
The following table shows the results on the CUB-200-2011 dataset. Best results are marked in bold.
| R@ | 1 | 10 | 20 | 30 | 40 | 50 |
|---|---|---|---|---|---|---|
| margin | 63.9 | 75.3 | 84.4 | 90.6 | 94.8 | - |
| HDC | 60.7 | 72.4 | 81.9 | 89.2 | 93.7 | 96.8 |
| HTL | 57.1 | 68.8 | 78.7 | 86.5 | 92.5 | 95.5 |
| A-BIER | 65.5 | 75.8 | 83.9 | 90.2 | 94.2 | 97.1 |
| ABE-8 | 60.6 | 71.5 | 79.8 | 87.4 | - | - |
| ------------- | ------------ | ------------ | ----------- | ----------- | ----------- | ----------- |
| Our Baseline | 73.1 | 81.9 | 87.6 | 91.4 | 93.8 | 96.2 |
| ABIR w/o SE-block | 77.5 | 84.1 | 88.7 | 91.7 | 94.2 | 96.3 |
| ABIR with SE-block | 78.1 | 84.6 | 88.7 | 91.8 | 94.4 | 96.6 |
The following table shows the results on the Stanford Online Products dataset. Best results are marked in bold.
| R@ | 1 | 10 | 100 | 1000 |
|---|---|---|---|---|
| Contrastive | 42.0 | 58.2 | 73.8 | 89.1 |
| Triplet | 42.1 | 63.5 | 82.5 | 94.8 |
| LiftedStruct | 62.1 | 79.8 | 91.3 | 97.4 |
| HDC | 69.5 | 84.4 | 92.8 | 97.7 |
| HTL | 74.8 | 88.3 | 94.8 | 98.4 |
| A-BIER | 74.2 | 86.9 | 94.0 | 97.8 |
| ABE-8 | 76.3 | 88.4 | 94.8 | 98.2 |
| ------------- | ------------ | ------------ | ----------- | ----------- |
| Our Baseline | 71.2 | 85.6 | 93.5 | 97.7 |
| ABIR w/o SE-block | 74.3 | 87.4 | 94.6 | 98.3 |
| ABIR with SE-block | 74.8 | 87.7 | 95.0 | 98.5 |
The following table shows the results on the Cars-196 dataset. Best results are marked in bold.
| R@ | 1 | 2 | 4 | 8 | 16 | 32 |
|---|---|---|---|---|---|---|
| margin | 86.9 | 92.7 | 95.6 | 97.6 | 98.7 | - |
| HDC | 83.8 | 89.8 | 93.6 | 96.2 | 97.8 | 98.9 |
| HTL | 81.4 | 88.0 | 92.7 | 95.7 | 97.4 | 99.0 |
| A-BIER | 82.0 | 89.0 | 93.2 | 96.1 | - | - |
| ABE-8 | 85.2 | 90.5 | 94.0 | 96.1 | - | - |
| ------------- | ------------ | ------------ | ----------- | ----------- | ----------- | ----------- |
| Our Baseline | 82.6 | 88.1 | 92.4 | 95.3 | 97.4 | 98.4 |
| ABIR w/o SE-block | 89.1 | 93.1 | 95.4 | 97.2 | 98.3 | 99.1 |
| ABIR with SE-block | 89.4 | 93.3 | 95.6 | 97.1 | 98.2 | 99.0 |
If this code helps your research, please cite our paper:
@inproceedings{nie2019weakly,
title={Weakly Supervised Image Retrieval via Coarse-scale Feature Fusion and Multi-level Attention Blocks},
author={Nie, Xinyao and Lu, Hong and Wang, Zijian and Liu, Jingyuan and Guo, Zehua},
booktitle={Proceedings of the 2019 on International Conference on Multimedia Retrieval},
pages={48--52},
year={2019},
organization={ACM}
}