/image-crop-analysis

Code for reproducing our analysis in the paper titled: Image Cropping on Twitter: Fairness Metrics, their Limitations, and the Importance of Representation, Design, and Agency

Primary LanguageJupyter NotebookApache License 2.0Apache-2.0

Image Crop Analysis

Open All Collab Binder DOI:10.1145/3479594 arxiv:2105.08667

How does a saliency algorithm work

This is a repo for the code used for reproducing our Image Crop Analysis paper as shared on our blog post.

If you plan to use this code please cite our paper as follows:

@article{TwitterImageCrop2021,
  author = {Yee, Kyra and Tantipongpipat, Uthaipon and Mishra, Shubhanshu},
  title = {Image Cropping on Twitter: Fairness Metrics, Their Limitations, and the Importance of Representation, Design, and Agency},
  year = {2021},
  issue_date = {October 2021},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  volume = {5},
  number = {CSCW2},
  url = {https://doi.org/10.1145/3479594},
  doi = {10.1145/3479594},
  journal = {Proceedings of the ACM on Human-Computer Interaction},
  month = oct,
  articleno = {450},
  numpages = {24},
  keywords = {image cropping, ethical HCI, fairness in machine learning, demographic parity, representational harm}
}
@article{TwitterImageCrop2021ArXiv,
       author = {{Yee}, Kyra and {Tantipongpipat}, Uthaipon and {Mishra}, Shubhanshu},
        title = "{Image Cropping on Twitter: Fairness Metrics, their Limitations, and the Importance of Representation, Design, and Agency}",
      journal = {arXiv e-prints},
     keywords = {Computer Science - Computers and Society, Computer Science - Computer Vision and Pattern Recognition, Computer Science - Human-Computer Interaction, Computer Science - Machine Learning},
         year = 2021,
        month = may,
          eid = {arXiv:2105.08667},
        pages = {arXiv:2105.08667},
archivePrefix = {arXiv},
       eprint = {2105.08667},
 primaryClass = {cs.CY},
}

Analysis of demographic bias of the image cropping algorithm

Instructions

Docker Run

  • Install docker
  • Run the following commands in this root directory of this project:
docker build -t "image_crop" -f docker/Dockerfile .
docker run -p 9000:9000 -p 8900:8900 -it image_crop
  • Open the jupyter lab URL shown in terminal.

Run on Google Colab

Open All Collab

  • Open a google colab notebook
  • Run the following code in the cell where HOME_DIR variable is set:
try:
    import google.colab
    ! pip install pandas scikit-learn scikit-image statsmodels requests dash
    ! [[ -d image-crop-analysis ]] || git clone https://github.com/twitter-research/image-crop-analysis.git
    HOME_DIR = Path("./image_crop_analysis").expanduser()
    IN_COLAB = True
except:
    IN_COLAB = False

Security Issues?

Please report sensitive security issues via Twitter's bug-bounty program (https://hackerone.com/twitter) rather than GitHub.