Advanced-RAG-Series

This repository contains advanced LLM-based chatbots for Retrieval Augmented Generation (RAG) and Q&A with different databases. (VectorDB, GraphDB, SQLite, CSV, XLSX, etc.). The repository provides guide on using both AzureOpenAI and OpenAI API for each project.

List of projects:

Q&A-and-RAG-with-SQL-and-TabularData
KnowledgeGraph-Q&A-and-RAG-with-TabularData

General structure of the projects:

Project-folder
  ├── README.md           <- The top-level README for developers using this project.
  ├── HELPER.md           <- Contains extra information that might be useful to know for executing the project.
  ├── .env                <- dotenv file for local configuration.
  ├── .here               <- Marker for project root.
  ├── configs             <- Holds yml files for project configs
  ├── explore             <- Contains my exploration notebooks and the teaching material for YouTube videos. 
  ├── data                <- Contains the sample data for the project.
  ├── src                 <- Contains the source code(s) for executing the project.
  |   └── utils           <- Contains all the necessary project modules. 
  └── images              <- Contains all the images used in the user interface and the README file.

NOTE: This is the general structure of the projects, however there might be small changes duo to the specific needs of each project.

Key Notes:

Key Note 1: All the project uses Azure OpenAI. So, to use OpenAI API directly, just change the credentials and switch the models completions.

Key Note 2 : When we interact with databases using LLM agents, good informative column names can help the agents to navigate easier through the database.

Key Note 3: When we interact with databases using LLM agents, remember to NOT use the database with WRITE privileges. Use only READ and limit the scope. Otherwise your user can manupulate the data (e.g ask your chain to delete data).

Key Note 4: Familiarity with database query languages such as Pandas for Python, SQL, and Cypher can enhance the user's ability to ask more better questions and have a richer interaction with the graph agent.

Project description:

Q&A-and-RAG-with-SQL-and-TabularData:

Q&A-and-RAG-with-SQL-and-TabularData is a chatbot project that utilizes GPT 3.5, Langchain, SQLite, and ChromaDB and allows users to interact (perform Q&A and RAG) with SQL databases, CSV, and XLSX files using natural language.

Features:

Chat with SQL data.
Chat with preprocessed CSV and XLSX data.
Chat with uploaded CSV and XSLX files during the interaction with the user interface.
RAG with Tabular datasets.

Databases:

Diabetes dataset: Link
Cancer dataset: Link
Chinook SQL database: Link

YouTube video: Link

KnowledgeGraph-Q&A-and-RAG-with-TabularData:

KnowledgeGraph-Q&A-and-RAG-with-TabularData is a chatbot project that utilizes knowledge graph, GPT 3.5, Langchain graph agent, and Neo4j graph database and allows users to interact (perform Q&A and RAG) with Tabular databases (CSV, XLSX, etc.) using natural language. This project also demonstrates an approach for constructing the knowledge graph from unstructured data by leveraging LLMs.