Measuring & visualising the environmental impact of LLMs using the Impact Framework 🌲💻💡

Created over an intense four week period as part of the Green Software Foundation Carbon Hack 2024 (Best Contribution Award 🏆)
Our project aims to enhance the Impact Framework by providing a visual element to complement the low-level infrastructure of the Impact Framework, in order to effectively communicate its results to a wider range of audiences.
We are a team of developers, AI enthusiasts and story tellers who share a keen interest and passion for sustainablity and the technical solutions that can help move towards a more sustainable IT industry.
Team made up of members of the Capgemini Insights & Data Sustainability Guild.

How to Use 🛠️

The Impact Framework output is to be copied into the output.json file.
This output.json file then connects to the website we have created to update six time series charts.
In the time we have had available to us in this hackathon we have not been able to automate the process of passing the Impact Framework plugin results to the website. This is something we will look to achieve in phase two of development!

Here is a link to the website we have created as part of our project:

Examples of the time series charts that our website will display:

Following the success of ChatGPT (and co.) there has been a significant increase in interest in LLMs by businesses across various sectors, which is predicted to continue to grow.
This is happening at the same time as businesses are experiencing an increase in sustainability reporting requirements 🔗 EU Corporate Sustainability Reporting Directive
There is rising concern about the vast amounts of energy that will be required to power the rise of LLMs (and AI more generally), which Sam Altman (ChatGPT creator 🤖) referenced in his recent speech as Davos 🔗 Nuclear Fusion needed to power superhuman AI - News
We have identified a potential gap between these two trends which could be filled with a tool like the Impact Framework.
Our project is inspired by the need to balance the benefits of the continued rise and development of AI (particularly LLMs) with the environmental impact of the massive amount of computing power needed.
We hope our project will inform and educate.

The website contains six time series charts showing the below metrics over a one hour time period:

The project workflow:

This template provides a minimal setup to get React working in Vite with HMR and some ESLint rules.

Currently, two official plugins are available:

Whilst creating and planning our project we wanted to ensure our contribution was aligned with the Impact Framework's Design Philosophy:

Transparency: We used existing IF plugins as a basis of our project and took advantage of the transparent manifest file structure. Our contribution is also transparent, with all steps we have taken being freely available for all to review on this Github repo.
Verifiability: Again, we have used the existing manifest file structure in our project to ensure our results are verifiable. We have designed the project so that numerous LLM use cases can be tested; from RAG to text generation to classification.
Flexibility: We have tested our project on a number of different LLM use cases (as noted above) and have six different outputs which users can review as needed for their use case. This provides flexibility in approach, whether a company is focused more on carbon or energy use.
Modularity: The solution we have created is highly modular and scalable.
Neutrality: The output we have created aims to inform only - not to provide any particular assessment on whether an LLM is suitable. Through the use of Prometheus, users will be able to add an alert for when any metric exceeds a certain value, but this value is not built into our solution.

We have also created a demo video which goes through our project on an end to end basis:

➡️➡️➡️ Video Demo!

@JadeMCap 👩‍💻 @BaldelliMichele 👨‍💻 @chivchila 👩‍💻 @wispyplant 👩‍💻 @arun-apad 👨‍💻 @ogbk 👨‍💻 @blazarblast 👨‍💻

Currently, the plugin's output is separate from the visuals. To update the website with new output, we manually replace the JSON file on the GitHub page and rerun the process.
Future goal: Automate the process by storing the output on a server. The website will then pull from this server, making the process more automated and easier to use.