/DukeMTMC-reID_evaluation

ICCV2017 The Person re-ID Evaluation Code for DukeMTMC-reID Dataset (Including Dataset Download)

Primary LanguageMatlabMIT LicenseMIT

DukeMTMC-reID Description

What's new: We updated the name of the dataset from 'Duke' to 'DukeMTMC-reID', added the original license from DukeMTMC and removed the redistribution limitation.

DukeMTMC-reID is a subset of the DukeMTMC for image-based re-identification, in the format of the Market-1501 dataset. The original dataset contains 85-minute high-resolution videos from 8 different cameras. Hand-drawn pedestrain bounding boxes are available.

We crop pedestrain images from the videos every 120 frames, yielding in total 36,411 bounding boxes with IDs. There are 1,404 identities appearing in more than two cameras and 408 identities (distractor ID) who appear in only one camera. We randomly select 702 IDs as the training set and the remaining 702 IDs as the testing set. In the testing set, we pick one query image for each ID in each camera and put the remaining images in the gallery.

As a result, we get 16,522 training images of 702 identities, 2,228 query images of the other 702 identities and 17,661 gallery images (702 ID + 408 distractor ID).

About Dataset

File Description
/bounding_box_test The gallery images. We retrieve a query from this image pool.
/bounding_box_train The training images. This dir contains the images from 702 different identities.
/query The query images. Each of them is from different identities in different cameras.

Naming Rule of the images In bbox "0005_c2_f0046985.jpg", "0005" is the identity. "c2" means the image from Camera 2. "f0046985" is the 46985th frame in the video of Camera 2.

Dataset Licence

Please follow the LICENSE_DukeMTMC-reID. You are free to share, create and adapt the DukeMTMC-reID dataset, in the manner specified in the license.

We also include the LICENSE_DukeMTMC. If you want to share, create and adapt the DukeMTMC dataset, please follow this license.

The DukeMTMC-reID evaluation code is under the MIT License.

Download Dataset

You can download the DukeMTMC-reID dataset from GoogleDriver or (BaiduYun password: chu1).

Some unzip tools on Windows may meet some problems. Please check that you have the following files after unzip:

If download links are unavailable, please don't hesitate to contact me to update links. Thank you.

Dataset Insights

  • Data Distribution

Figure. The image distribution of DukeMTMC-reID training set. We note that the median of images per ID is 20. But some ID may contain lots of images, which may comprise some algorithms. (For example, ID 5388 contains 426 images.)

Thank Xun for suggestions.

  • Camera Topology

This picture is from DukeMTMC Homepage.

Evaluation

To evaluate, you need to calculate your gallery and query feature (i.e., 17661x2048 and 2228x2048 matrix) and save them in advance. Then download the codes in this repository. You just need to change the image path and the feature path in the evaluation_res_duke_fast.m and run it to evaluate.

State-of-the-art

Methods Rank@1 mAP Reference
BoW+kissme 25.13% 12.17% "Scalable person re-identification: a benchmark", Liang Zheng, Liyue Shen, Lu Tian, Shengjin Wang, Jingdong Wang and Qi Tian, ICCV 2015 [project]
LOMO+XQDA 30.75% 17.04% "Person Re-identification by Local Maximal Occurrence Representation and Metric Learning", Shengcai Liao, Yang Hu, Xiangyu Zhu and Stan Z Li, CVPR 2015 [project]
Basel. 65.22% 44.99% "Person Re-identification: Past, Present and Future", Liang Zheng, Yi Yang, and Alexander G. Hauptmann, arXiv:1610.02984 [code]
Basel. + LSRO   67.68% 47.13% "Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in vitro", Zhedong Zheng, Liang Zheng and Yi Yang, ICCV 2017 [code]
Basel. + OIM 68.1% - "Joint Detection and Identification Feature Learning for Person Search", Tong Xiao, Shuang Li, Bochao Wang, Liang Lin, Xiaogang Wang, CVPR 2017
Verif + Identif 68.9% 49.3% "A Discriminatively Learned Cnn Embedding for Person Re-identification", Zhedong Zheng, Liang Zheng, and Yi Yang, TOMM 2017. [code]
APR 70.69% 51.88% "Improving person re-identification by attribute and identity learning", Yutian Lin, Liang Zheng, Zhedong Zheng, Yu Wu, Yi Yang, arXiv:1703.07220 [Attribute Dataset]
ACRN 72.58% 51.96% "Person Re-Identification by Deep Learning Attribute-Complementary Information", Arne Schumann and Rainer Stiefelhagen, CVPR 2017 Workshop
PAN 71.59% 51.51% "Pedestrian Alignment Network for Large-scale Person Re-identification", Zhedong Zheng, Liang Zheng, Yi Yang, arXiv:1707.00408 [code]
PAN+rerank 75.94% 66.74%
FMN 74.51% 56.88% "Let Features Decide for Themselves: Feature Mask Network for Person Re-identification", Guodong Ding, Salman Khan, Zhenmin Tang, Fatih Porikli, arXiv:1711.07155
FMN+rerank 79.52% 72.79%
SVDNet 76.7% 56.8% "SVDNet for Pedestrian Retrieval", Yifan Sun, Liang Zheng, Weijian Deng, Shengjin Wang, ICCV 2017 [code]
DPFL 79.2% 60.6% "Person Re-Identification by Deep Learning Multi-Scale Representations", Yanbei Chen, Xiatian Zhu and Shaogang Gong, ICCV2017 workshop
SVDNet + REDA 79.31% 62.44% "Random Erasing Data Augmentation", Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, Yi Yang, arXiv:1708.04896 [code]
SVDNet + REDA + ReRank 84.02% 78.28%
Mid-level Representation 80.43% 63.88% "The Devil is in the Middle: Exploiting Mid-level Representations for Cross-Domain Instance Matching", Qian Yu, Xiaobin Chang, Yi-Zhe Song, Tao Xiang, Timothy M. Hospedales, arXiv:1711.08106
Deep-Person 80.90% 64.80% "Deep-Person: Learning Discriminative Deep Features for Person Re-Identification", Xiang Bai, Mingkun Yang, Tengteng Huang, Zhiyong Dou, Rui Yu, Yongchao Xu, arXiv:1711.10658
PSE 79.8% 62.0% "A Pose-Sensitive Embedding for Person Re-Identification with Expanded Cross Neighborhood Re-Ranking", M. Saquib Sarfraz, Arne Schumann, Andreas Eberle, Rainer Stiefelhagen, arXiv:1711.10378 [code]
PSE + ECN + ReRank 85.2% 79.8%
PCB 83.3% 69.2% "Beyond Part Models: Person Retrieval with Refined Part Pooling", Yifan Sun, Liang Zheng, Yi Yang, Qi Tian, Shengjin Wang, arXiv:1711.09349

Baseline

We release our baseline training code and pretrained model in [Matconvnet Version] and [Pytorch Version]. You can choose one of the two tools to conduct the experiment.

Or you can directly download the finetuned ResNet-50 baseline feature. You can download it from GoogleDriver or BaiduYun, which includes the feature of training set, query set and gallery set. The DukeMTMC-reID LICENSE is also included.

Sample Retrieval

DukeMTMC-attribute

We also annotated 23 human-level attributes (gender/clothing/...) for DukeMTMC-reID. You can find it in the following link: https://github.com/vana77/DukeMTMC-attribute

Citation

DukeMTMC Dataset [Bibtex]

DukeMTMC-reID Dataset, Protocol, Baseline [Bibtex]