/olaris-llm

Nuvolaris LLM provider

Primary LanguageShell

Nuvolaris LLM Plugin

Install

This is a nuv plugin to setup and manage an LLM provider for Nuvolaris.

To use, install nuv and clone the plugin for development:

git clone https://github.com/nuvolaris/olaris-llm

Then you have a nuv llm subcommand (if you run from the parent folder)

Use

Setup a few convenience aliases:

alias nlg="nuv llm gcp"
alias nlu="nuv llm util"
alias nlt="nuv llm tgi"

Prerequisites

You need:

  • the gcloud command
  • a gcloud account,
  • billing and authorizations set up
  • a default project
  • vm with a public hostname to connect from

Manage a Cloud VM with GPU

Commands:

  • nlg create: create a VM with GPU

  • nlg destroy: destroy a VM with GPU

  • nlg start: start the VM with GPU

  • nlg stop: stop the VM with GPU

  • nlg enter: enter in the VM

  • nlg use: star, enter in the vm, then stop it when you exit

Build tgi

nlt build