rexterity/JARVIS

Your own personal voice assistant: Voice to Text to LLM to Speech, displayed in a web interface

JARVIS

Your own voice personal assistant: Voice to Text to LLM to Speech, displayed in a web interface.

How it works

1. 🎤 The user speaks into the microphone
2. ⌨️ Voice is converted to text using Deepgram
3. 🤖 Text is sent to OpenAI's GPT-3 API to generate a response
4. 📢 Response is converted to speech using ElevenLabs
5. 🔊 Speech is played using Pygame
6. 💻 Conversation is displayed in a webpage using Taipy

Video Demo

Requirements

Python 3.8 - 3.11

Make sure you have the following API keys:

• Deepgram
• OpenAI
• Elevenlabs

How to install

1. Clone the repository

git clone https://github.com/AlexandreSajus/JARVIS.git

2. Install the requirements

pip install -r requirements.txt

3. Create a .env file in the root directory and add the following variables:

DEEPGRAM_API_KEY=XXX...XXX
OPENAI_API_KEY=sk-XXX...XXX
ELEVENLABS_API_KEY=XXX...XXX

How to use

1. Run display.py to start the web interface

python display.py

2. In another terminal, run jarvis.py to start the voice assistant

python main.py

• Once ready, both the web interface and the terminal will show Listening...
• You can now speak into the microphone
• Once you stop speaking, it will show Stopped listening
• It will then start processing your request
• Once the response is ready, it will show Speaking...
• The response will be played and displayed in the web interface.

Here is an example:

Listening...
Done listening
Finished transcribing in 1.21 seconds.
Finished generating response in 0.72 seconds.
Finished generating audio in 1.85 seconds.
Speaking...

 --- USER: good morning jarvis
 --- JARVIS: Good morning, Alex! How can I assist you today?

Listening...
...