Badness.ai is an open-source, community-operated catalog of harms involving generative AI, built in pursuit of safer, more responsible AI systems.
Generative AI is an exciting platform shift that will transform our lives. But it also comes with profound implications for society. We believe that a healthy dose of caution, sobriety, and pragmatism toward generative AI are crucial for developing safer, more robust AI systems.
Badness.ai is scoped to examples of harm involving generative AI, not AI and machine learning more broadly. Generative AI increases the space of things that machine learning models can say or do by many orders of magnitude. This inflection not only unlocks proportional business value but also scales up the surface area of opportunities for badness. By focusing on generative AI, we can help mitigate these systemic risks in the early days before they are baked into the technologies we use.
Examples of generative AI badness are everywhere; on the news, Twitter, on various forums, and elsewhere across the internet. Examples can be of people abusing generative AI to cause harm, or harm initiated by the systems themselves. Examples can be from demonstrative settings or empirical behavior out in the wild. Badness.ai is a systematic catalog of all these stories, assembled by — and for — the community.
By thoughtfully documenting and curating past failures of generative AI systems, we hope we can empower model developers and application developers to take a more proactive approach to safety — and not repeat past mistakes. We equally hope that our catalog aides policymakers, journalists, and the public in understanding and responding to the unique safety challenges of generative AI. We hope to do so without falling into the trap of sensationalism, fearmongering, or information overload.
Badness.ai is inspired by the AI Incidents Database, PrivacySpy, and a number of other projects. We are grateful for these projects and the inspiration they have provided.
Badness.ai is an open source project, and we expect our purpose and scope to evolve over time in partnership with the broader AI safety community.
Aspirationally, this website will be the most high-quality and accessible catalog of verified harm caused by AI systems. We can only achieve this ambitious goal with help from contributors like you. This page lays out the values of our project, various ways you can contribute, and guidelines for our content.
Our project is guided by the following values:
- Inclusivity — Harms of AI systems often disproportionally affect marginalized and underresourced groups; it is crucially important that our project — and our contributor community — reflect this reality.
- Grounding — We do not speculate about hypothetical future harms of AI systems, as legitimate as those harms might be. This website is a catalog of past harms. We must keep our work grounded in actual, present harms.
- Optimism — We are optimistic about the positive impacts of AI, both now and in the future. We simply believe that the best way to build safer and more responsible AI systems is to learn from the mistakes of the past. This site is not "anti-AI" — not by a long shot.
- Reliability — Our project relies on well-researched, verifiable instances of harm. We do not accept anecdotal evidence. It is crucial that our catalog of harms is as accurate as possible. More information about our sourcing and verification process is below.
There are a number of ways you can contribute to the project.
If you have a suggestion, found an issue, or want to share an idea, please feel free to create a GitHub issue on our repository.
To add a new incident to the catalog, please do so using a GitHub pull request. Incidents of harm are stored in the catalog/ directory; simply add a new file there. You can use the existing incidents as a reference. Titles should be a single full sentence, and descriptions should be about four full sentences split into paragraphs as needed. Please use active voice whenever possible in your incident titles and descriptions, and maintain consistency with the verb tense (incidents that happened in the past should use past tense, and ongoing incidents should use present tense).
All contributions to the project are subject to review by the project maintainers. We will do our best to respond to all contributions in a timely manner. If you have any questions about the project, please feel free to reach out to us via GitHub or email.
We expect all contributors to follow our code of conduct, which is modeled after the Contributor Covenant. All contributions are licensed under the Apache 2.0 license.