/tabby-backend-py

Tabby (self-hosted AI coding assistant) server in 20 lines of python

Primary LanguagePythonApache License 2.0Apache-2.0

Tabby server in 20 lines of python

Tabby is a self-hosted OSS AI coding assistant
I use it every day and you should too
Please support the authors https://github.com/TabbyML/tabby

The problem I have with it is the complexity of its backend
So I reverse engineered it in python (without reading the codebase, some bugs are likely)

Installation

Clone this repo and install 2 dependencies:

git clone https://github.com/vsolina/tabby-backend-py.git
cd tabby-backend-py
# optionally create venv:
python3 -m venv venv && source venv/bin/activate
pip install -r requirements.txt

Download the model you want to use, e.g. 🤗:

wget https://huggingface.co/TabbyML/StarCoder-1B/resolve/main/ggml/q8_0.v2.gguf

Update model path in app.py (literally called model_path in model_config)
Use relative or absolute path to the model (no shell expansion, e.g.: ~/)

Run the service:

python app.py

# OR if you want to customize the port, host
flask run --port 8080 --host 0.0.0.0

Install Tabby Extension and update server URL in the Extension configuration:
https://tabby.tabbyml.com/docs/extensions/

Customization:

Notes:

Long live the king Georgi https://github.com/ggerganov

Start unnecessary Rant 👴 🤌 ☁️

We must fight the complexity demon because it wins by default
I believe software should be anticomplex

Original TabbyML server implementation (v0.7.0)
has 530 dependencies (https://github.com/TabbyML/tabby/blob/main/Cargo.lock)
has 7178 lines of Rust code in 73 files across 9 crates
uses 3 languages (Rust, Python, C++) according to github

This implementation has
2 (first level) dependencies
20 lines of code in one file (app-min.py), 671 bytes

simple is better than complex is better than complicated

When I can read it
I can understand it
I can fix it

That makes it antifragile and easy to maintain

Rant over, have a great day