This project aims to implement a document-based question-answering system using a local LLM model, Python, and the Langchain Framework. It processes PDF documents, breaking them into ingestible chunks, and then stores these chunks into a Chroma DB vector database for querying. It complements a Medium article called Harnessing Local Language Models - A Guide to Transitioning From OpenAI to On-Premise Power.
These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.
To install the project, you need to have Python installed on your machine. Also you need a machine with a cuda compatible GPU. If you want to run the application on the CPU, you need to change the cuda references in the code to cpu.
The project uses several dependencies. After cloning the repository, navigate to the project directory and install dependencies with the following commands:
pip install -r requirements
To ingest documents, place your PDF files in the 'docs' folder make sure that you are in the app folder and run the following command:
cd app
python ingest.py
To query the ingested documents, make sure that you are in the app folder, run the following command and follow the interactive prompts:
cd app
python query.py
This project is licensed under the MIT license - see the LICENSE.md file for details