RAG using Llama3, Langchain and ChromaDB
This project utilizes Llama3 Langchain and ChromaDB to establish a Retrieval Augmented Generation (RAG) system. This system empowers you to ask questions about your documents, even if the information wasn't included in the training data for the Large Language Model (LLM). Retrieval Augmented Generation works by first performing a retrieval step when presented with a question. This step fetches relevant documents from a special vector database, where the documents have been indexed.
- LLM: Large Language Model
- Llama3: LLM developed by Meta
- Langchain: Framework designed to streamline the creation of applications utilizing LLMs
- Vector database: Database that organizes data using high-dimensional vectors
- ChromaDB: Vector database
- RAG: Retrieval Augmented Generation (see below for more details)
- Model: Llama 3
- Variation: 8b-chat-hf (8b: 8 Billion parameters; hf: HuggingFace)
- Version: V1
- Framework: Transformers
The pre-trained Llama3 model is fine-tuned with over 15 Trillion tokens and boasts 8 to 70 Billion parameters, making it one of the most powerful open-source models available. It offers significant advancements over the previous Llama2 model.
This project successfully implemented a Retrieval Augmented Generation (RAG) solution by leveraging Langchain, ChromaDB, and Llama3 as the LLM. To evaluate the system's performance, we utilized the EU AI Act from 2023. The results demonstrated that the RAG model delivers accurate answers to questions posed about the Act.
Future Work
To further enhance the solution, we will focus on refining the RAG implementation. This will involve optimizing the document embeddings and exploring the use of more intricate RAG architectures.
