Dalai Pi

Run LLaMA and Alpaca on your computer.

Info

This is a fork of Dalai to add a ChatGPT-style UI to it.
Also supports Alpaca 30B, you will need to download it manually from here

Please consider giving this project a star if you like it. It would motivate me to work on the fork and improve it further.

Screenshots

Requirements

RAM requirements

7B: ~5 GB
13B: ~9 GB
30B: ~21 GB
65B: ~42 GB

Disk Space Requirements

Alpaca:

7B: 4.21GB (pre-quantized) 13B: 8.14GB (pre-quantized)

LLaMA

Note: after quantizing the model, you can delete the rest of the files and keep only the quantized model. 7B

Full: The model takes up 31.17GB
Quantized: 4.21GB

13B

Full: The model takes up 60.21GB
Quantized: 4.07GB * 2 = 8.14GB

30B

Full: The model takes up 150.48GB
Quantized: 5.09GB * 4 = 20.36GB

65B

Full: The model takes up 432.64GB
Quantized: 5.11GB * 8 = 40.88GB

Get started

Mac

Install Homebrew

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Install dependencies with Homebrew

brew install cmake
brew install pkg-config
brew install git
brew install yarn

Download Dalai-Pi

git clone https://github.com/ItsPi3141/dalai ~/dalai_pi

Install required Nodejs modules

cd ~/dalai_pi
yarn install

Download LLaMA or Alpaca models

Note: replace "llama" with "alpaca" if you want Alpaca. Change "7B" to the model size that you want. Alpaca only has 7B and 13B while LLaMA has 7B, 13B, 30B, and 65B.

npx dalai llama install 7B

Start Dalai-Pi web UI

npx dalai serve

After the web UI has opened, go to http://localhost:42069 and enjoy!

Windows

Install Visual Studio by downloading and running this script dalai-install-vs2019-only-win.bat

IMPORTANT
On Windows, make sure to run all commands in cmd.
DO NOT run in powershell. Powershell has unnecessarily strict permissions and makes the script fail silently.

Install Git from here

Note: Make sure you check the add to PATH option
Download Dalai-Pi

git clone https://github.com/ItsPi3141/dalai %USERPROFILE%\dalai_pi

Install required Nodejs modules

cd %USERPROFILE%\dalai_pi
npm install yarn -g
yarn install

Download LLaMA or Alpaca models

Note: replace "llama" with "alpaca" if you want Alpaca. Change "7B" to the model size that you want. Alpaca only has 7B and 13B while LLaMA has 7B, 13B, 30B, and 65B.

npx dalai llama install 7B

Start Dalai-Pi web UI

.\webui-start.bat

Linux

Install dependencies Ubuntu/Debian:

sudo apt update
sudo apt install build-essential python3-venv -y

Fedora/CentOS:

sudo dnf upgrade --refresh
sudo dnf install -y make automake gcc gcc-c++ kernel-devel python3-virtualenv

Install Python and Node.js Ubuntu/Debian:

cd ~
curl -sL https://deb.nodesource.com/setup_18.x -o /tmp/nodesource_setup.sh
sudo bash /tmp/nodesource_setup.sh
sudo apt install nodejs python3 python-is-python3 -y

Fedora/CentOS:

sudo dnf install -y curl
curl -fsSL https://rpm.nodesource.com/setup_18.x | sudo bash -
sudo dnf install -y nodejs python python3-pip python-numpy

Download Dalai-Pi

git clone https://github.com/ItsPi3141/dalai ~/dalai_pi

Install required Nodejs modules

cd ~/dalai_pi
yarn install

Download LLaMA or Alpaca models

Note: replace "llama" with "alpaca" if you want Alpaca. Change "7B" to the model size that you want. Alpaca only has 7B and 13B while LLaMA has 7B, 13B, 30B, and 65B.

npx dalai llama install 7B

Start Dalai-Pi web UI

npx dalai serve

After the web UI has opened, go to http://localhost:42069 and enjoy!

Staying up to date

Have questions or feedback? Follow the project through the following outlets:

GitHub Twitter Discord