What if you could ask a question and get an answer from the Monte Carlo documentation?
This repo is an implementation of a locally hosted chatbot specifically focused on question answering over the Monte Carlo documentation. Built with OpenAI ChatGPT API, LangChain and FastAPI.
The app leverages LangChain's streaming support and async API to update the page in real time for multiple users.
Note: this app requires OpenAI's ChatGPT API. It costs money to use the API. More information here: OpenAI ChatGPT
- Clone the repo:
git clone git@github.com:mvfolino68/MonteCarloGPT.git
- Install dependencies: tested on Python 3.10.9
- from within the repo, setup a virtual environment:
python3 -m venv _venv
- activate the virtual environment:
source _venv/bin/activate
- install dependencies:
pip install -r requirements.txt
- from within the repo, setup a virtual environment:
- Set up an OpenAI API Key:
- you can use the OpenAI API documentation to set up an API key.
- set the environment variables for your API key. See Environment Variables for more information.
- Setup a vectorstore:
- option 1: Use the existing vectorstore:
- download the vectorstore from here and place it in the root directory of the repo. download here
- option 2: Create a new vectorstore:
- Note: this method will take a while to run due to the size of the Monte Carlo docs.
- run
python ingest.py
to ingest Monte Carlo docs data into the vectorstore (only needs to be done once). - you can use other Document Loaders to load your own data into the vectorstore.
- option 1: Use the existing vectorstore:
pip install uvicorn[standard]
so your local machine has websockets (beyond the requirements.txt)- Run the app:
make start
- Open localhost:9000 in your browser.
- Ask a question! 🎉
You can set these environment variables in a .env
file in the root directory of the project. See the .env.example
file for an example.
Variable | Description | Link |
---|---|---|
OPENAI_API_KEY |
Your OpenAI API key. | OpenAI Authentication |
OPENAI_API_TYPE |
"open_ai" if using OpenAI API. |
- |
OPENAI_API_BASE |
Leave this blank if using OpenAI API directly. | - |
AZURE_OPENAI_DEPLOYMENT_NAME |
Leave this blank if using OpenAI API directly. | - |
AZURE_OPENAI_MODEL |
Leave this blank if using OpenAI API directly. | - |
OPENAI_API_VERSION |
Leave this blank if using OpenAI API directly. | - |
Variable | Description | Link |
---|---|---|
OPENAI_API_KEY |
Your Azure OpenAI API key. | Azure OpenAI Quickstart |
OPENAI_API_TYPE |
Set this to "azure" if using Azure OpenAI. |
- |
OPENAI_API_BASE |
The base URL for your Azure OpenAI resource. | Azure OpenAI Quickstart |
AZURE_OPENAI_DEPLOYMENT_NAME |
The name of your Azure OpenAI deployment. | - |
AZURE_OPENAI_MODEL |
The name of the model you are using. | Azure OpenAI Quickstart |
OPENAI_API_VERSION |
The OpenAI API version and should align with the model you are using. | Azure OpenAI Quickstart |
Variable | Description | Link |
---|---|---|
PINECONE_API_KEY |
Your Pinecone API key. | Pinecone Quickstart |
PINECONE_ENVIRONMENT |
The name of your Pinecone environment. | Pinecone Quickstart |
PINECONE_INDEX |
The name of the Pinecone index. | Pinecone Quickstart |
- Install pgvector in your postgres
a. https://github.com/pgvector/pgvector#installation
a.
CREATE EXTENSION vector;
- Set the environment variables
This method requires that you have Docker installed on your machine. To install Docker, follow the instructions here. This setup assumes that you set up your environment variables in a .env
file in the root directory of the project.
docker build -t montecarlogpt .
docker run --env-file .env -p 9000:9000 montecarlogpt
- Pull html from the Monte Carlo documentation using
sitemap.xml
and BeautifulSoup to clean the html. - Load the data into DocumentStore using LangChain's UstructuredHTML Loader.
- Chunk the documents into smaller chunks using LangChain's TextSplitter.
- Create embeddings for each chunk using OpenAI's OpenAI Embeddings.
- Load the embeddings into a vectorstore using LangChain's vectorstore wrapper.
- FAISS is used as the vectorstore in this example. More information on FAISS can be found here.
- User accesses text input box and chat history via the web app.
- The web app sends the chat history and user input to the backend. The backend uses LangChain's ConversationalRetrievalChain to:
- Determine what a standalone question would be (using ChatGPT).
- Look up relevant documents from the vectorstore.
- Pass the standalone question and relevant documents to ChatGPT to generate a final answer.
- Return the final answer to the web app and add the answer to the chat history.
-
If you receive an error like this:
WARNING:/Users/<user>/Documents/MonteCarloGPT/.venv/lib/python3.10/site-packages/langchain/chat_models/openai.py:Retrying langchain.chat_models.openai.acompletion_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised APIConnectionError: Error communicating with OpenAI.
-
If you receive this error when using the download vectorstore:
ERROR:root:IndexFlat.search() missing 3 required positional arguments: 'k', 'distances', and 'labels'
Rerun the
ingest.py
script to create a new vectorstore.
Blog Posts:
I'm happy to accept contributions to this project. Please open an issue or a pull request.
- Make the app available online (currently only available locally). This will require a hosting service that can support the vectorstore and the web app.
- Allow users to input their own OpenAI API key in the frontend.
- Include more documentation sites.
- Tune the ChatVectorDBChain to improve the quality of the answers.
- Produce the chat history as a topic.
Twitter: @mvfolino68 LinkedIn: mfolino