/SDGZSL

[ICCV2021] Official Pytorch implementation for SDGZSL (Semantics Disentangling for Generalized Zero-Shot Learning)

Primary LanguagePython

Semantics Disentangling for Generalized Zero-Shot Learning

This is the official implementation for paper

Zhi Chen, Yadan Luo, Ruihong Qiu, Zi Huang, Jingjing Li, Zheng Zhang.
Semantics Disentangling for Generalized Zero-Shot Learning
International Conference on Computer Vision (ICCV) 2021.

Semantics Disentangling for Generalized Zero-shot Learning


Supplementary Experimental Results

In the paper, we followed the datasets provided in [15], in which the visual features in FLO dataset are normalized, and the semantic description of CUB dataset is the CNN-RNN sentence embeddings (1024D). We hereby provide extra comparison results on the visual features provided by GBU setting [17].

Model AwA2 T1 u s H aPY T1 u s H CUB-EMB T1 u s H CUB-ATT T1 u s H
LFGAA [4] 68.1 27.0 93.4 41.9 - - - - - - - - 67.6 36.2 80.9 50.0
DVBE [7] - 63.6 70.8 67.0 - 32.6 58.3 41.8 - - - - - 53.2 60.2 56.5
DVBE* [7] - 62.7 77.5 69.4 - 37.9 55.9 45.2 - - - - - 64.4 73.2 68.5
f-CLS WGAN[1] 65.3 56.1 65.5 60.4 40.5 32.9 61.7 - - 50.3 58.3 54.0 57.3 43.7 57.7 49.7
CANZSL[8] 68.9 49.7 70.2 58.2 - - - - - - - - 60.6 47.9 58.1 52.5
LisGAN [9] 70.6 52.6 76.3 62.3 43.1 34.3 68.2 45.7 - - - - 58.8 46.5 57.9 51.6
CADA-VAE[10] 64.0 55.8 75.0 63.9 - 31.7 55.1 40.3 - 52.0 54.8 53.4 61.8 51.6 53.5 52.4
f-VAEGAN-D2[11] 71.1 57.6 70.6 63.5 - - - - - - - - 61.0 48.4 60.1 53.6
f-VAEGAN-D2*[11] 70.3 57.1 76.1 65.2 - - - - - - - - 72.9 63.2 75.6 68.9
DLFZRL [12] 70.3 - - 60.9 46.7 - - 38.5 - - - - 61.8 - - 51.9
TF-VAEGAN [13] 72.2 59.8 75.1 66.6 - - - - - - - - 64.9 52.8 64.7 58.1
TF-VAEGAN*[13] 73.4 55.5 83.6 66.7 - - - - - - - - 74.3 63.8 79.3 70.7
E-PGN [15] 73.4 52.6 83.5 64.6 - - - - 72.4 52.0 61.1 56.2 - - - -
AGZSL [16] 73.8 65.1 78.9 71.3 41.0 35.1 65.5 45.7 - - - - 57.2 41.4 49.7 45.2
AGZSL*[16] 76.4 52.6 86.5 76.8 43.7 36.2 58.6 44.8 - - - - 77.2 69.2 76.4 72.6
SDGZSL 72.1 64.6 73.6 68.8 45.4 38.0 57.4 45.7 75.5 59.9 66.4 63.0 62.8 51.5 58.7 54.9
SDGZSL* 74.3 69.6 86.5 73.7 47.0 36.2 60.7 47.5 78.5 73.0 77.5 75.1 73.7 66.0 75.9 70.6
Model FLO-GBU T1 u s H FLO-EPGN T1 u s H SUN T1 u s H
LFGAA [4] - - - - - - - - 61.5 18.5 40.0 25.3
DVBE [7] - - - - - - - - - 45.0 37.2 40.7
DVBE* [7] - - - - - - - - - 44.1 41.6 42.8
f-CLSWGAN[1] 67.2 59.0 73.8 65.6 - - - - 60.8 42.6 36.6 39.4
CANZSL[8] 69.7 58.2 77.6 66.5 - - - - 60.1 46.8 35.0 40.0
LisGAN [9] 69.6 57.7 83.8 68.3 - - - - 61.7 42.9 37.8 40.2
CADA-VAE[10] 65.2 51.6 75.6 61.3 - - - - 61.8 47.2 35.7 40.6
f-VAEGAN-D2[11] 67.7 56.8 74.9 64.6 - - - - 64.7 45.1 38.0 41.3
f-VAEGAN-D2*[11] 70.4 63.3 92.4 75.1 - - - - 65.6 50.1 37.8 43.1
DLFZRL [12] - - - - - - - - 61.3 - - 42.5
TF-VAEGAN [13] 70.8 62.5 84.1 71.7 - - - - 66.0 45.6 40.7 43.0
TF-VAEGAN*[13] 74.7 63.8 92.5 79.4 - - - - 66.7 41.8 51.9 46.3
E-PGN [15] - - - - 85.7 71.5 82.2 76.5 - - - -
AGZSL [16] - - - - 82.7 63.5 94.0 75.7 63.3 29.9 40.2 34.3
AGZSL*[16] - - - - 86.9 73.7 91.9 81.7 66.2 50.5 43.1 46.5
SDGZSL 73.3 62.2 79.3 69.8 85.4 83.3 90.2 86.6 62.4 48.2 36.1 41.3
SDGZSL* 76.6 73.2 88.7 80.2 86.9 86.1 89.1 87.8 65.2 51.1 40.2 45.0

TODO

  • Results on CUB with attributes
  • Results on FLO without normalization
  • Results on SUN
  • Release the code of supplementary experiments

Requirements

The implementation runs on

  • Python 3.6

  • torch 1.3.1

  • Numpy

  • Sklearn

  • Scipy

Usage

Put your datasets in SDGZSL_data folder and run the scripts in the folder.

The extracted features for APY and AWA datasets are from [1], FLO and CUB datasets are from [2]. For the fine-tuned features, AWA,FLO and CUB are from [3]. The APY fine-tuned features are extracted from us.

[1] Xian, Yongqin, et al. "Feature generating networks for zero-shot learning." CVPR 2018.

[2] Yu, Yunlong, et al. "Episode-based prototype generating network for zero-shot learning." CVPR 2020.

[3] Narayan, Sanath, et al. "Latent embedding feedback and discriminative features for zero-shot classification." ECCV 2020.

[4] Y. Liu, et al. "Attribute attention for semantic disambiguation in zero-shot learning." CVPR 2019.

[7] S. Min, et al. "Domain-aware visual bias eliminating for generalized zeroshot learning." CVPR 2020.

[8] Z. Chen, et al, "CANZSL: Cycleconsistent adversarial networks for zero-shot learning from natural language," WACV, 2020.

[9] J. Li, et al. "Leveraging the invariant side of generative zero-shot learning." CVPR, 2019.

[10] E. Schonfeld, et al. "Generalized zero-and few-shot learning via aligned variational autoencoders." CVPR, 2019.

[11] Y. Xian, et al. "f-vaegan-d2:A feature generating framework for any-shot learning." CVPR, 2019.

[12] B. Tong, et al. "Hierarchical disentanglement of discriminative latent features for zero-shot learning." CVPR, 2019.

[13] S. Narayan, et al. "Latent embedding feedback and discriminative features for zero-shot classification." ECCV, 2020.

[15] Y. Yu, et al. "Episode-based prototype generating network for zero-shot learning." CVPR, 2020.

[16] C., Yu-Ying, et al. "Adaptive and generative zero-shot learning." ICLR, 2020.

[17 ] Y. Xian, et al. "Zero-shot learning—a comprehensive evaluation of the good, the bad and the ugly." TPAMI, 2018.

Citation:

If you find this useful, please cite our work as follows:

@inproceedings{chen2021semantics,
	title={Semantics Disentangling for Generalized Zero-shot Learning},
	author={Chen, Zhi and Luo, Yadan and Qiu, Ruihong and Huang, Zi and Li, Jingjing and Zhang, Zheng},
	booktitle={ICCV},
	year={2021}
}