bezaleel22/rust-llm-rag

To implement LLM as a services.

🦀🦀🦀 Rust LLM Included the RAG

This project is to shake the RUST off of my machine learning skills and learn about the emerging technologies in this era, particularly large language models (LLMs).

📹 Demo Video

📐 Architecture

🚀 Tech Stack

• Qdrant
• Axum
• Socketoxide
• Ollama

👩‍🏭 LLM Model

• dolphin-mistral:latest

🧙‍♂️ Embedding Model

• nomic-ai/nomic-embed-text-v1

🎢 Collection Distacing

• Cosine

🧠 What is RAG ???

The RAG (Retrieval-Augmented Generation) is the process of optimizing the output of a large language model, so it references an authoritative knowledge base outside of its training data sources before generating a response.

In other words, when you need a LLM to remember historical information, you engage in a conversation where your inputs are stored in a vector database. In subsequent interactions, you retrieve related historical data from this database, combine it with your current prompt, and use this enhanced prompt to continue the conversation with the model.

📑 References

• What RAG is
• Qdrant
• Building a RAG Web Service with Qdrant and Rust