Spin-O-Llama

Ollama api implementation for spin

⚠️ Proof of concept: This project is not production ready

Quick Start

Install spin
login to fermeyon cloud
```
spin login
```

clone this repository

git clone https://github.com/BLaZeKiLL/Spin-O-Llama.git
cd Spin-O-Llama

build
```
spin build
```
deploy
```
spin deploy
```

Routes implemented

POST /api/generate

supported request body

{
    "model": "<supported-model>",
    "prompt": "<input prompt>",
    "system": "<system prompt>", // optional, system prompt
    "stream": false, // streaming not supported, has no impact
    "options": { // optional, llm options
        "num_predict": 128,
        "temperature": 0.8,
        "top_p": 0.9,
        "repeat_penalty": 1.1
    } // default values provided above
}

response body

{
    "model": "<model-id>",
    "response": "<output>",
    "done": true
}

POST /api/embeddings

supported request body

{
    "model": "<model-id>", // doesn't matter for now will always use all-minilm-l6-v2
    "prompt": "<input>"
}

response body

{
    "embedding": [<float array>]
}

Model compatibility

generate - llama2-chat, codellama-instruct
embeddings - all-minilm-l6-v2

Contributing

Contributions are welcome for further implementation of the Ollama api that is supported on the spin runtime.