/llm-inference-platform

Dev repo for HuggingFace inference platform.

Primary LanguagePythonMIT LicenseMIT

LLM inference platform

CI Check Markdown links Documentation Status CD

Repository for tools to easily deploy LLMs on Princeton's della cluster.

Current status

We provide an easy command line interface to deploy LLMs (such as LLaMA) on della. All functionality can be accessed installation-free in the browser using the mydella JupyterHub interface.

Usage

See tutorial.

Vision & Roadmap

It should be as easy for researchers to use an open-source LLM as a closed/proprietary LLM.

Read more about our vision and roadmap.

Development installation

Installation

pip3 install --editable '.[dev,test,docs]'

Please also install the pre-commit hook:

pipx run pre-commit install

Alternatively, you can run pre-commit manually with nox -s lint.

In addition, nox provides the following:

  • To run addition python lint checks, run nox -s pylint
  • To build the documentation, run nox -s docs (the resulting documentation will be rendered at docs/_build/html/index.html)