RAG with Llama3 on Groq

This cookbook shows how to do retrieval-augmented generation (RAG) using Llama3 on Groq.

For embeddings we can either use Ollama or OpenAI.

Note: Fork and clone this repository if needed

1. Create a virtual environment

python3 -m venv ~/.venvs/aienv
source ~/.venvs/aienv/bin/activate

2. Export your Groq API Key

export GROQ_API_KEY=***

3. Use Ollama or OpenAI for embeddings

Since Groq doesnt provide embeddings yet, you can either use Ollama or OpenAI for embeddings.

To use Ollama for embeddings Install Ollama and run the nomic-embed-text model

ollama run nomic-embed-text

To use OpenAI for embeddings, export your OpenAI API key

export OPENAI_API_KEY=sk-***

4. Install libraries

pip install -r cookbook/llms/groq/rag/requirements.txt

5. Run PgVector

Install docker desktop first.

Run using a helper script

./cookbook/run_pgvector.sh

OR run using the docker run command

docker run -d \
  -e POSTGRES_DB=ai \
  -e POSTGRES_USER=ai \
  -e POSTGRES_PASSWORD=ai \
  -e PGDATA=/var/lib/postgresql/data/pgdata \
  -v pgvolume:/var/lib/postgresql/data \
  -p 5532:5432 \
  --name pgvector \
  phidata/pgvector:16

6. Run RAG App

streamlit run cookbook/llms/groq/rag/app.py

Open localhost:8501 to view your RAG app.
Add websites or PDFs and ask question.
Example Website: https://techcrunch.com/2024/04/18/meta-releases-llama-3-claims-its-among-the-best-open-models-available/
Ask questions like:
- What did Meta release?
- Tell me more about the Llama 3 models?

fznumber/asistente