Mattermost AI Framework
Table of Contents
Background
This project is a framework for a self-hosted AI app in a multi-user chat environment that can be fully private and off-grid AKA air-gapped. Check out the demo from May 15, 2023.
This framework uses a locally-deployed Mattermost app to interface with a variety of LLM AIs. It currently supports local LLMs hosted via Serge, a wrapper around llama.cpp that can run LLMs without a GPU.
This framework consists of three local components:
- Mattermost
- Serge
ai-bot
, a Mattermost app inside the./ai-bot
folder
ai-bot
routes communicaiton between the Mattermost and Serge servers via a REST API.
Install
Local
You will need Docker installed with compose
. This repository should work on a 16GB M1 Macbook.
- Clone and enter this repository:
git clone https://github.com/mattermost/mattermost-ai-framework && cd mattermost-ai-framework
- Start the services:
docker compose up -d
- Download a Serge model (e.g., GPT4All):
- Open Serge at
http://localhost:8008
- Select Download Models
- Download GPT4All and wait for it to finish
- Access Mattermost
- Open Mattermost at
http://localhost:8065
- Select View in Browser
- Create your local account and team
- Install the
ai-bot
Mattermost app
- In any Mattermost channel, use this slash command:
/apps install http http://ai-bot:9000/manifest.json
- Accept the permissions in the modal
- Select Submit
- If unable to complete the above steps, try restarting the app service first:
docker restart ai-bot
Gitpod
- Select the above badge to start your Gitpod workspace
- The workspace will configure itself automatically. Wait for the services to start and for your
root
login for Mattermost to be generated in the terminal - Download a Serge model (e.g., GPT4All):
- Check for blocked pop-ups, or open Serge on the
Ports
tab. - Select Download Models
- Download GPT4All and wait for it to finish
- Access Mattermost and log in with the generated
root
credentials - Install the
ai-bot
Mattermost app
- In any Mattermost channel, use this slash command:
/apps install http http://ai-bot:9000/manifest.json
- Accept the permissions in the modal
- Select Submit
- If unable to complete the above steps, try restarting the app service first:
docker restart ai-bot
You're now ready to use the example ai-bot
!
Usage
Ask a question
In any channel, you can now ask ai-bot
questions with the /ai ask
slash command. For example:
/ai ask "Write a haiku about perseverance"
/ai ask "What is the weather right now in Glasgow?"
/ai ask "When were pterodactyls alive?"
Slash command | Response |
---|---|
Summarize thread
To summarize threads, first grant the bot account access to public channels:
- Open the top left Mattermost menu button (9 squares) and select Integrations
- Select Bot Accounts then Edit for
ai-bot
- Check the box for post:channels (Bot will have access to post to all Mattermost public channels)
Now, open the message app menu button (4 squares) on any post in a public channel and select Summarize (AI). You can watch a brief demo of this functionality here.
Message app menu button | Response |
---|---|
Related Efforts
The current stage is an early proof-of-concept. There are several directions this could go in, including:
- Support for other AI/LLM services
- Additional bot capabilities
- Improved model handling
- Better UX
Explore Mattermost's AI initiatives:
- https://docs.mattermost.com/about/mattermost-customizable-chatgpt-bot-framework.html
- https://mattermost.com/add-chatgpt-to-mattermost/
- https://github.com/Brightscout/mattermost-plugin-openai
- https://github.com/yGuy/chatgpt-mattermost-bot
Contributing
See Mattermost's contributor guide to learn about contributing to our open source projects like this one.
License
This repository is licensed under Apache-2.