Born from a conversation with James Governor at Redmonk, we discussed the potential of having a document to outline what's inside any given model. I'd personally noticed differences in accuracy scores from using different programming languages (R, Java and Python) with even the most traditional algorithms like Decision Trees, Multiple Linear Regression and Support Vector Machines.
I settled on a Bibtex like structure for this proof of concept bill of materials (BOM) for any AI or Machine Learning model. The aim for it to be easily read by machine or human.
The AI Bill of Materials (BoM) Schema serves as a comprehensive blueprint to document the various components involved in creating and deploying artificial intelligence systems. With the advent of increasingly complex AI models and systems, maintaining transparency and traceability in development processes is of utmost importance. This schema aims to bridge that gap.
The primary objectives of the AI BoM Schema are:
- Transparency: Providing clarity on the tools, hardware, data sources, and methodologies used in the development of AI systems.
- Reproducibility: Offering enough information for researchers and developers to reproduce the models and results.
- Accountability: Ensuring creators and users of AI systems are aware of their origins, components, and performance metrics.
- Ethical and Responsible AI: Encouraging the documentation of training data sources, including any synthetic data used, to ensure there's knowledge about potential biases, limitations, or ethical considerations.
- Hardware and Software Details: Information about the computational hardware used for training and inference, programming languages involved, and ML library versions.
- Data Source Documentation: Comprehensive details about training and validation datasets, including the classification and details of synthetic data.
- Performance Metrics: Essential metrics like R2 scores, F1 scores, and others that give insights into the model's performance.
- Training History: A chronological record of the model's training, allowing tracking of its evolution.
In line with our commitment to promoting responsible AI, the schema will undergo continuous refinement. The roadmap for its development includes:
- Integration with Common Platforms: Developing plugins or extensions for popular AI and ML platforms to facilitate easy adoption of the BoM schema.
- Public Feedback Loop: Engaging the wider AI community for feedback to enhance and refine the schema.
- Case Studies: Publishing case studies demonstrating the application and benefits of the BoM schema in real-world scenarios.
- Ethical Guidelines: Collaborating with ethics researchers to ensure the schema aligns with the latest responsible AI guidelines and standards.
We encourage contributions from the community! Whether you're a developer, researcher, ethicist, or AI enthusiast, your insights can help shape the future of this schema towards more ethical and transparent AI systems.
Please refer to the CONTRIBUTING.md for guidelines on how to contribute.
This project is licensed under the MIT License. See the LICENSE.md file for details.
With the rapid advancements in AI, having a structured, transparent, and comprehensive record like the AI Bill of Materials is a step towards ensuring AI evolves responsibly. We're excited to see the community's engagement and the positive changes it will bring.