/LLM_API

A short API for loading models for inference from other apps/computers

Primary LanguagePythonApache License 2.0Apache-2.0

LLM_API

A short API for loading models for inference from other apps/computers

Install FastAPI and its dependencies:

pip install fastapi
pip install uvicorn

Launch with:

uvicorn app:app --reload