aida-ugent/llm-ideology-analysis

Public repository for the paper "Large Language Models Reflect the Ideology of their Creators"

LLM Ideology Analysis

This repository contains the code and analysis tools for the paper "Large Language Models Reflect the Ideology of their Creators". We provide a comprehensive framework for analyzing ideological biases in Large Language Models (LLMs) through their evaluations of historical political figures.

📊 Dataset

The dataset contains evaluations from 17 different LLMs of 4,339 political figures, with responses in both English and Chinese. Access the full dataset on Hugging Face.

📚 Citation

@misc{buyl2024largelanguagemodelsreflect,
      title={Large Language Models Reflect the Ideology of their Creators}, 
      author={Maarten Buyl and Alexander Rogiers and Sander Noels and Iris Dominguez-Catena and Edith Heiter and Raphael Romero and Iman Johary and Alexandru-Cristian Mara and Jefrey Lijffijt and Tijl De Bie},
      year={2024},
      eprint={2410.18417},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2410.18417}, 
}

👥 Team

Authors

• Maarten Buyl (*‡) - Ghent University, Belgium
• Alexander Rogiers (†) - Ghent University, Belgium
• Sander Noels (†) - Ghent University, Belgium
• Iris Dominguez-Catena - Public University of Navarre, Spain
• Edith Heiter - Ghent University, Belgium
• Raphael Romero - Ghent University, Belgium
• Iman Johary - Ghent University, Belgium
• Alexandru-Cristian Mara - Ghent University, Belgium
• Jefrey Lijffijt - Ghent University, Belgium
• Tijl De Bie - Ghent University, Belgium

* Corresponding author: maarten.buyl@ugent.be
† These authors contributed equally to this work
‡ Lead author

Affiliations

• Ghent University
  Department of Electronics and Information Systems
  IDLab
  Technologiepark-Zwijnaarde 122
  9052 Ghent, Belgium

• Public University of Navarre
  Department of Statistics, Computer Science and Mathematics
  31006 Pamplona, Spain

📧 Contact

For questions or issues, please:

1. Open an issue in this repository
2. Contact one of the corresponding authors: maarten.buyl@ugent.be, alexander.rogiers@ugent.be or sander.noels@ugent.be