- MySQL Database for statistics (optional)
- Node.js
- Instance of Ollama (local or on Akash)
- Docker
-
Install all dependencies
-
Download font
- Manually download and paste the font from Fontshare.
- Download Family -> Extract ZIP -> WEB -> paste files and insert them into the frontend/src/scss/fonts/vendor/satoshi folder.
-
Deploy Ollama instance
- The text is generated by one or more Ollama instances running on Akash, but you can also use a local instance.
3.1. Deploy on Akash
- The easiest option is to deploy Ollama on Akash. Use this SDL for deployment.
3.2. Deploy locally
- To deploy locally, run the following command to automatically pull the latest Ollama Docker image and start it:
After completion, run different models with the command:
docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
Find a full list of models on the Ollama website.docker exec -it ollama ollama run mistral
-
Setup the GPU Load Balancer
- Ollama supports switching models on the go, and for load balancing, use HAProxy. Adjust the HAProxy config based on the number of instances. Build the Docker image for HAProxy in the backend/haproxy folder:
Start it with the command:
docker build -t my-haproxy .
This exposes the load balancer on port 3332 and the stats page on port 3333. Change the admin password in the config if used in production.docker run -d --name my-running-haproxy -p 3333:3333 -p 3332:3332 my-haproxy
- Ollama supports switching models on the go, and for load balancing, use HAProxy. Adjust the HAProxy config based on the number of instances. Build the Docker image for HAProxy in the backend/haproxy folder:
-
Setup MySQL DB (optional)
- For production logging of request amounts, set up a MySQL database. Create a table with the supplied database schema backend/logdb.sql. To disable logging, set
LOGDB
tofalse
in your .env file or environment.
- For production logging of request amounts, set up a MySQL database. Create a table with the supplied database schema backend/logdb.sql. To disable logging, set
-
Start/Build the Backend
- Navigate to the backend and run
yarn dev
to start it locally oryarn build
to build the source for a Akash deployment. To deploy to Akash, runpush it to a registry of your choice and create a deployment with it on akash. Make sure to set the exposed port (port 3001) in your deployment file. Set CORS_ORIGIN in your environment or .env to allow requests from your frontend.docker build -t yourusername/backend:version .
- Navigate to the backend and run
-
Start/Build the Frontend
- Open the .env.example file, rename it to .env, and set the variables to use the backend URL. Navigate to the frontend and run
yarn start
to start it locally oryarn build
for Akash deployment. To deploy to Akash, runand use the supplied SDL for the deployment.docker build -t yourusername/frontend:version .
- Open the .env.example file, rename it to .env, and set the variables to use the backend URL. Navigate to the frontend and run
Now you should be set up. If you encounter any problems, feel free to open an issue.