- 2020 Visual Dialog challenge winning submission.
This repository is the official implementation of Ensemble of MRR and NDCG models for Visual Dialog. (To appear in NAACL'21)
We propose a two-step non-parametric ranking approach that can merge strong MRR and NDCG models. Using our method, we manage to keep most MRR state-of-the-art performance (70.41% vs. 71.24%) and the NDCG state-of-the-art performance (72.16% vs. 75.35%).
The approach is a rank ensemble of:
- The MRR model: Factor Graph Attention
- The NDCG model: Large-scale Pretraining for Visual Dialog: A Simple State-of-the-Art Baseline
Place the following meta-data under the data dir: https://netapp-my.sharepoint.com/:f:/p/idans/ElgrBd3pZfNKox6qhMkC5qoBLOvQYktEzd4eoVOZg_3Kcw?e=75gQTT
Place the following models' scores under models dir: https://netapp-my.sharepoint.com/:f:/p/idans/EqZ6gMPLk5BIvyLdLqp4L2QBsiyU4iG6BC6D7vviNILRmg?e=Pub1JC
Please follow the jupyter notebook: Two-step ranks ensemble.ipynb.