KITT

KITT is a ChatGPT-powered AI that lives in a WebRTC conference call.

Online Demo

You can try an online demo right now at https://livekit.io/kitt

How it works

This repo contains two services:

meet
lkgpt-service

The meet service is a NextJS app that implements a typical video call app. The lkgpt-service implements KITT. When a room is created, a webhook calls a handler in lkgpt-service which adds a participant to the room. The particpant uses GCP's speech-to-text, ChatGPT, and GCP's text-to-speech to create KITT.

The following diagram illustrates this:

Getting started

Prerequisites

GOOGLE_APPLICATION_CREDENTIALS json body from a Google cloud account. See https://cloud.google.com/docs/authentication/application-default-credentials#GAC
OpenAI api key
LiveKit api_key, api_secret, and url from LiveKit Cloud
Go 1.19+ and Node.js

Running Locally

To run locally, you'll need to run the two services in this repo: meet and lkgt-service.

Running Meet Locally:

In the meet/ directory, copy .env.example to .env.local and fill in your LiveKit connection details. Then run:

# From the meet/ directory
yarn dev

Running lkgpt-service Locally:

# From the lkgpt-service/ directory
go run /cmd/server/main.go --config config.yaml --gcp-credentials-path gcp-credentials.json`

Once both services are running you can navigate to http://localhost:3000. There's one more step needed when running locally. When deployed, KITT is spawned via a LiveKit webhook, but locally - the webhook will have no way of reaching your local lkgpt-service that's running. So you'll have to manually call an API to spawn KITT: