adrianmarino/gpt4-quantized

gpt4-quantized

gpt4all-lora-quantized-ggml

Run locally a quantized GTP4 version with 4096 characters context.

Also use a langchan agent to use external tools to help model to answer questions.

Requirements

• miniconda or mamba

Setup

Step 1: Download model from Here.

Step 2: creatae ./models directory and move gpt4all-lora-quantized-ggml.bin model file to it.

Step 3

$ conda env create -f environment.yml

Step 4

$ conda activate gpt

Step 5

$ jupyter lab

Jupyter Notebook 6.1.4 is running at:
http://localhost:8888/?token=45efe99607fa6......

Step 6: Go to jupyter url and run model.ipynb or agent.ipynb

References

• llama.cpp
• LangChain