/awesome-vlm-and-society

VLM and Society

Recent papers on Vision-Language Models' bias/stereotype issues.

Stereotype and Bias

  • [Paper] DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generative Models, Jaemin Cho, Abhay Zala, Mohit Bansal, February 2022
  • [Paper] Red-Teaming the Stable Diffusion Safety Filter, Javier Rando, Daniel Paleka, David Lindner, Lennart Heim, Florian Tramèr, October 2022
  • [Paper] Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale, Federico Bianchi, Pratyusha Kalluri, Esin Durmus, Faisal Ladhak, Myra Cheng, Debora Nozza, Tatsunori Hashimoto, Dan Jurafsky, James Zou, Aylin Caliskan, November 2022 (FAccT'23)
  • [Paper] Fair Diffusion: Instructing Text-to-Image Generation Models on Fairness, Felix Friedrich, Manuel Brack, Lukas Struppek, Dominik Hintersdorf, Patrick Schramowski, Sasha Luccioni, Kristian Kersting, February 2023
  • [Paper] Stable Bias: Analyzing Societal Representations in Diffusion Models, Alexandra Sasha Luccioni, Christopher Akiki, Margaret Mitchell, Yacine Jernite, March 2023
  • [Paper] T2IAT: Measuring Valence and Stereotypical Biases in Text-to-Image Generation, Jialu Wang, Xinyue Gabby Liu, Zonglin Di, Yang Liu, Xin Eric Wang, June 2023
  • [Paper] Evaluating the Social Impact of Generative AI Systems in Systems and Society, Irene Solaiman, Zeerak Talat, William Agnew, Lama Ahmad, Dylan Baker, Su Lin Blodgett, Hal Daumé III, Jesse Dodge, Ellie Evans, Sara Hooker, Yacine Jernite, Alexandra Sasha Luccioni, Alberto Lusoli, Margaret Mitchell, Jessica Newman, Marie-Therese Png, Andrew Strait, Apostol Vassilev, June 2023

Cross-Cultural Difference

  • [Paper] Exploiting Cultural Biases via Homoglyphs in Text-to-Image Synthesis, Lukas Struppek, Dominik Hintersdorf, Felix Friedrich, Manuel Brack, Patrick Schramowski, Kristian Kersting, September 2022
  • [Paper] Towards Equitable Representation in Text-to-Image Synthesis Models with the Cross-Cultural Understanding Benchmark (CCUB) Dataset, Zhixuan Liu, Youeun Shin, Beverley-Claire Okogwu, Youngsik Yun, Lia Coleman, Peter Schaldenbrand, Jihie Kim, Jean Oh, January 2023
  • [Paper] On the Cultural Gap in Text-to-Image Generation, Bingshuai Liu, Longyue Wang, Chenyang Lyu, Yong Zhang, Jinsong Su, Shuming Shi, Zhaopeng Tu, July 2023
  • [Paper] Navigating Cultural Chasms: Exploring and Unlocking the Cultural POV of Text-To-Image Models, Mor Ventura, Eyal Ben-David, Anna Korhonen, Roi Reichart, October 2023

Surveys

  • [Paper] Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements, Jiawen Deng, Jiale Cheng, Hao Sun, Zhexin Zhang, Minlie Huang, February 2023
  • [Paper] On the Trustworthiness Landscape of State-of-the-art Generative Models: A Survey and Outlook, Mingyuan Fan, Chengyu Wang, Cen Chen, Yang Liu, Jun Huang, July 2023
  • [Paper] Trustworthy Large Models in Vision: A Survey, Ziyan Guo, Li Xu, Jun Liu, November 2023