From Static to Dynamic: Adapting Landmark-Aware Image Models for Facial Expression Recognition in Videos
From Static to Dynamic: Adapting Landmark-Aware Image Models for Facial Expression Recognition in Videos
Yin Chen$^{†}$, Jia Li$^{†∗}$, Shiguang Shan, Meng Wang, and Richang Hong
[2024.9.5] The fine-tuned checkpoints are available.
[2024.9.2] The code and pre-trained models are available.
[2024.8.28] The paper is accepted by IEEE Transactions on Affective Computing.
[2023.12.5] Code and pre-trained models will be released here.
1、 Download the pre-trained weights from baidu drive or google drive, and move it to the ckpts directory.
2、 Run the following command to fine-tune the model on the target dataset.
conda create -n s2d python=3.9
conda activate s2d
pip install -r requirements.txt
bash run.shThe fine-tuned checkpoints can be downloaded from here.
| Datasets | w/o oversampling | w/ oversampling | ||
|---|---|---|---|---|
| UAR | WAR | UAR | WAR | |
| FERV39K | ||||
| FERV39K | 41.28 | 52.56 | 43.97 | 46.21 |
| DFEW | ||||
| DFEW01 | 61.56 | 76.16 | 64.80 | 75.35 |
| DFEW02 | 59.93 | 73.99 | 62.54 | 72.53 |
| DFEW03 | 61.33 | 76.41 | 66.47 | 75.87 |
| DFEW04 | 62.75 | 76.31 | 66.03 | 74.48 |
| DFEW05 | 63.51 | 77.27 | 67.43 | 76.80 |
| DFEW | 61.82 | 76.03 | 65.45 | 74.81 |
| MAFW | ||||
| MAFW01 | 32.78 | 46.76 | 36.16 | 44.21 |
| MAFW02 | 40.43 | 55.96 | 41.94 | 51.22 |
| MAFW03 | 47.01 | 62.08 | 48.08 | 61.48 |
| MAFW04 | 45.66 | 62.61 | 47.67 | 60.64 |
| MAFW05 | 43.45 | 59.42 | 43.16 | 58.55 |
| MAFW | 41.86 | 57.37 | 43.40 | 55.22 |
If you find this work helpful, please consider citing:
@ARTICLE{10663980,
author={Chen, Yin and Li, Jia and Shan, Shiguang and Wang, Meng and Hong, Richang},
journal={IEEE Transactions on Affective Computing},
title={From Static to Dynamic: Adapting Landmark-Aware Image Models for Facial Expression Recognition in Videos},
year={2024},
volume={},
number={},
pages={1-15},
keywords={Adaptation models;Videos;Computational modeling;Feature extraction;Transformers;Task analysis;Face recognition;Dynamic facial expression recognition;emotion ambiguity;model adaptation;transfer learning},
doi={10.1109/TAFFC.2024.3453443}}