Setting up the chatbot locally:
- Download the model from huggingface (https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF), you can choose any model, I have chosen the
mistral-7b-instruct-v0.1.Q4_K_M.gguf
because of my machine. - Make
models
folder in the backend directory and save the downloaded model there. - Also make
data
folder in the backend directory. - In the backend directory run the requirements.txt file.
- Run the backend server (cd into the directory)
uvicorn main:app --reload
- Keep the backend server running and cd into the frontend directory.
- Build the frontend app using
npm install
and then run the app usingnpm start
- The app will run on
http://localhost:3000