Memo-GPT

A complete Software As a Service application that helps users take notes during meetings, providing summaries of recorded or uploaded audio, chat about the meeting or audio with a GPT-based AI and query the AI for specific answers within the audio without the necessity of listening to the entire audio.

This SAS application effectively eliminates the need for manual note-taking during meetings, lectures, and conversations, as it autonomously does it for you with intelligence.

VIDEO DEMO

This video demonstration emphasizes the essential features of the app using the Gradio interface. If you prefer to skip the recording process and go straight to the AI demo, feel free to jump to 3:10.

memogpt_main_demo.2.mp4

Memo-GPT is an advanced AI-powered application designed to enhance your meeting experiences. It offers audio recording, audio file uploading, transcript generation, and summarization. Additionally, it enables querying/chatting and provides responses based on the meeting content, complete with timestamps and speaker diarization. This tool aims to increase efficiency and elevate the outcomes of your meetings.

Features

Audio recording
- Start and stop recording
Audio file uploading
Speech transcription using AssemblyAI API
Text summarization using the Meta's LLama-2 API
Conversational question answering based on the transcript using Meta's LLama-2 70b API
User account management and authentication
Email verification on signup
Premium subscriptions
- Limits on transcription length for free accounts
- Unlimited transcription for paid accounts
Payment processing with Paystack
FRONTEND with Gradio

Technologies

Python
FastAPI
MySQL
SQLAlchemy
JWT Authentication
OAuth2 Password Flow
AssemblyAI API
LLama API
Paystack API
Gradio
ffmpeg

Folder Structure

├── database
│   ├── db.py
│   └── model
│       ├── transcript.py
│       └── users_model.py
├── schema
│   ├── transcript.py 
│   └── users_schema.py
├── services
│   ├── assemblyai_services.py
│   ├── audio_services.py
│   ├── history_services.py  
│   ├── llama_services.py
│   ├── premium_services.py
│   ├── storage_services.py
│   ├── tokenizer_services.py
│   └── user_services.py
├── tmp
├── .env
├── .gitignore
├── audio.py
├── email_setup.py
├── email_verification.py
├── hashing.py
├── history.py 
├── llama.py
├── main.py
├── main_gradio.py
├── oauth.py
├── paystack.py
├── prompts.py
├── README.md
├── token_key.py
├── user.py
└── user_login.py

Database Models

The main database models are:

User - Stores user account info
Audio - Stores audio files and transcripts
Summary - Transcript summaries generated by LLama
History - Conversation history with the LLama bot

Relationships are defined between user, audio, summaries and history.

API Endpoints

The API endpoints are organized by HTTP method below:

GET Endpoints

GET /verify/{token} - Verify email
GET /audio - Stream audio
GET /webhook_verification - Verify webhook
GET /audio/data - Get audio data
GET /all_chats - Get all chats

POST Endpoints

POST /signup - Create user
POST /login - Login user
POST /audio/update_data - Update audio
POST /pay - Process payment
POST /response - Get bot response
POST /audio/start - Start recording
POST /audio/play - Play audio
POST /audio/upload - Upload audio

PUT Endpoints

PUT /delete_account - Delete account

DELETE Endpoints

DELETE /delete_chat/{id} - Delete chat

Gradio UI

This project incorporates a Gradio user interface for demonstration and testing purposes. The objective is to facilitate comprehension of the project for non-technical individuals, without the necessity of interacting with the API endpoints.

Premium Accounts

To access transcripts over 5 minutes users need a premium account. This is handled via:

Premium pricing plans defined in users_schema.py
paystack.py handles payments with Paystack API
premium_services.py checks if user account is premium
Duration stored in User model to track when premium expires

Installation

Clone the repository

git clone https://github.com/user/meeting-assistant

Install dependencies

pip install -r requirements.txt

Set environment variables

Copy the .env.example file to .env and fill in values for:

DB_URL
AssemblyAI_API_KEY
LLama_API_KEY
Paystack_SECRET_KEY
SMTP_Credentials

Initialize the database

alembic upgrade head

Run Gradio frontend

python main_gradio.py

Run the server API

uvicorn main:app

The Gradio UI will be available at http://127.0.0.1:7860

The API will be available at http://localhost:8000

Contributing

Contributions are welcome! Please open an issue or PR if you would like to help add features or fix bugs.

DivineUX23/MemoGPT