/apparent_skincolor

"Beyond Skin Tone: A Multidimensional Measure of Apparent Skin Color" (ICCV 2023)

Primary LanguagePythonApache License 2.0Apache-2.0

Apparent Skin Color

Beyond Skin Tone: A Multidimensional Measure of Apparent Skin Color
William Thong, Przemyslaw Joniak, Alice Xiang
ICCV 2023

[Paper]

Skin color score extraction

  • Run extract/predict.py to extract skin color scores (i.e., perceptual lightness L* and hue angle h*)
    • Only PIL, scikit-image and scikit-learn are needed
    • Perceptual lightness L* is associated with skin tone (values below 60 are considered to be dark tones, while values above are light tones)
    • Hue angle h* is associated with skin hue (values below 55 are considered to red hues, while values above are yellow hues)
  • We also provide the results of this script in extract/results for the CelebAMask-HQ, FFHQ and CFD datasets
  • When masks are not available, we extract them with DeepLabV3 trained on CelebAMask-HQ

Experiments

Saliency-based image cropping

We evaluate the fairness of the open-source image cropping model from Twitter by comparing a pair of facial images. In an ideal scenario, the cropping model should have an equal preference for both faces.

Experiments are reported in Section 4.2.1 of the main paper, and are adapted from [Birhane et al, WACV 2022]. To reproduce them, follow these steps:

Face verification

We evaluate the fairness of face verification models.

Experiments are reported in Section 4.2.2 of the main paper.

Skin color causal effect in attribute prediction

We evaluate the causal effect of skin color in several (non)-commercial models. To achieve this, we modify the skin color of images in CelebAMask-HQ by moving in the latent space that are meaningful to edit the skin tone and the skin hue independently.

Citation

If you find this repository useful for your research, please consider citing our work:

@inproceedings{thong2023skincolor,
  title={Beyond Skin Tone: A Multidimensional Measure of Apparent Skin Color},
  author={Thong, William and Joniak, Przemyslaw and Xiang, Alice},
  booktitle={ICCV},
  year={2023}
}