The app has a React/Vite frontend and a FastAPI backend and model serve is use ray serve.
Run the backend (I use Poetry for package management - pip install poetry
if you don't have it):
cd backend
echo "OPENAI_API_KEY=sk-your-key" > .env
poetry install
poetry shell
poetry run uvicorn main:app --reload --port 8001
If you want to use Anthropic, add the ANTHROPIC_API_KEY
to backend/.env
with your API key from Anthropic.
Run the frontend:
cd frontend
yarn
yarn dev
Open http://localhost:5173 to use the app.
If you prefer to run the backend on a different port, update VITE_WS_BACKEND_URL in frontend/.env.local
Run model serve
cd modelserve
pip install -r requirements.txt
ray start --head
serve run ./config.yaml
If you have Docker installed on your system, in the root directory, run:
echo "OPENAI_API_KEY=sk-your-key" > .env
docker-compose up -d --build
The app will be up and running at http://localhost:5173. Note that you can't develop the application with this setup as the file changes won't trigger a rebuild.