/mattermost-ai-framework

The Mattermost AI Framework

Primary LanguageTypeScriptApache License 2.0Apache-2.0

Mattermost AI Framework

Open in Gitpod

standard-readme compliant

Screenshot

Table of Contents

Background

This project is a framework for a self-hosted AI app in a multi-user chat environment that can be fully private and off-grid AKA air-gapped. Check out the demo from May 15, 2023.

This framework uses a locally-deployed Mattermost app to interface with a variety of LLM AIs. It currently supports local LLMs hosted via Serge, a wrapper around llama.cpp that can run LLMs without a GPU.

This framework consists of three local components:

  1. Mattermost
  2. Serge
  3. ai-bot, a Mattermost app inside the ./ai-bot folder

ai-bot routes communicaiton between the Mattermost and Serge servers via a REST API.

Install

Local

You will need Docker installed with compose. This repository should work on a 16GB M1 Macbook.

  1. Clone and enter this repository:
  • git clone https://github.com/mattermost/mattermost-ai-framework && cd mattermost-ai-framework
  1. Start the services: docker compose up -d
  2. Download a Serge model (e.g., GPT4All):
  • Open Serge at http://localhost:8008
  • Select Download Models
  • Download GPT4All and wait for it to finish
  1. Access Mattermost
  • Open Mattermost at http://localhost:8065
  • Select View in Browser
  • Create your local account and team
  1. Install the ai-bot Mattermost app
  • In any Mattermost channel, use this slash command: /apps install http http://ai-bot:9000/manifest.json
  • Accept the permissions in the modal
  • Select Submit
  • If unable to complete the above steps, try restarting the app service first: docker restart ai-bot

Gitpod

Open in Gitpod

  1. Select the above badge to start your Gitpod workspace
  2. The workspace will configure itself automatically. Wait for the services to start and for your root login for Mattermost to be generated in the terminal
  3. Download a Serge model (e.g., GPT4All):
  • Check for blocked pop-ups, or open Serge on the Ports tab.
  • Select Download Models
  • Download GPT4All and wait for it to finish
  1. Access Mattermost and log in with the generated root credentials
  2. Install the ai-bot Mattermost app
  • In any Mattermost channel, use this slash command: /apps install http http://ai-bot:9000/manifest.json
  • Accept the permissions in the modal
  • Select Submit
  • If unable to complete the above steps, try restarting the app service first: docker restart ai-bot

You're now ready to use the example ai-bot! 🎉

Usage

Ask a question

In any channel, you can now ask ai-bot questions with the /ai ask slash command. For example:

  • /ai ask "Write a haiku about perseverance"
  • /ai ask "What is the weather right now in Glasgow?"
  • /ai ask "When were pterodactyls alive?"
Slash command Response

Summarize thread

To summarize threads, first grant the bot account access to public channels:

  1. Open the top left Mattermost menu button (9 squares) and select Integrations
  2. Select Bot Accounts then Edit for ai-bot
  3. Check the box for post:channels (Bot will have access to post to all Mattermost public channels)

Now, open the message app menu button (4 squares) on any post in a public channel and select Summarize (AI). You can watch a brief demo of this functionality here.

Message app menu button Response

Related Efforts

The current stage is an early proof-of-concept. There are several directions this could go in, including:

  • Support for other AI/LLM services
  • Additional bot capabilities
  • Improved model handling
  • Better UX

Explore Mattermost's AI initiatives:

Contributing

See Mattermost's contributor guide to learn about contributing to our open source projects like this one.

License

This repository is licensed under Apache-2.