2023 Hackathon for STEM Club
This project takes advantage of Cloudflare's new Workers AI, Meta LLaMA, OpenAI Whisper, and ElevenLabs.io.
When using the client, your voice will first be uploaded to the server, converted into an MP3 using ffmpeg, and transcribed into text using OpenAI Whisper.
After finding the raw text, it will then be run through LLaMA, to create a response to the query. This is where the PocketBase instance comes in, being used for storing message logs and chat history, for the AI to have conversation context.
Then, this response will be sent to ElevenLabs, in order to be processed as a natural sounding human voice to playback to the user.
For this project, you will need:
- A free Cloudflare account
- A free ElevenLabs.io account
- A computer or cloud server with NodeJS 16+ and ffmpeg installed
- A PocketBase instance, run locally, or hosted (ex. pockethost.io)
This monorepo is organized into a two folders.
This is the SvelteKit client used to access Apex.
This is the server to be run on your personal computer, or a cloud server.
Note that ELEVENLABS_API is optional, but it will restrict the ability for Apex to talk.
ELEVENLABS_API=
CLOUDFLARE_ACCOUNT=
CLOUDFLARE_API=
PB_EMAIL=
PB_PASSWORD=This is the URL used to access your server.
Note that VITE_INSPARE_MUSIC_KEY is optional, but it will restrict music playback.
VITE_API_URL=
VITE_INSPARE_MUSIC_KEY=