Ollama Nix Flake

A flake following the main branch of ollama. It's purpose is to build the most recent version supporting new models until the version in nixpkgs is updated.

Install
- Version
- Backend
Update
Remove
License

Install

You need to have nix flakes enabled; if you don't, see the nixos wiki for information on flakes and enabling them.

Install ollama to user profile:

nix profile install github:abysssol/ollama-flake

Create a temporary shell with ollama:

nix shell github:abysssol/ollama-flake

Use as an input in another flake:

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";

    ollama.url = "github:abysssol/ollama-flake";
    #ollama.inputs.nixpkgs.follows = "nixpkgs"; # this could break the build unless using unstable nixpkgs
  };

  outputs = { nixpkgs, ollama, ... }:
    let
      system = abort "system needs to be set";
      # to access the rocm package of the ollama flake:
      ollama-rocm = ollama.packages.${system}.rocm;
      #ollama-rocm = inputs'.ollama.packages.rocm; # with flake-parts

      pkgs = nixpkgs.legacyPackages.${system};
      # you can override package inputs like with nixpkgs
      ollama-cuda = ollama.packages.${system}.cuda.override { cudaGcc = pkgs.gcc11; };
    in
    {
      # output attributes go here
    };
};

Version

You can specify a version by appending /version after the main url, where version is any branch or tag. The version branches will only be updated in a backward compatible way, no breaking changes (see semantic versioning).

Append /1 to follow branch 1 which tracks version 1.y.z of the repo:

nix profile install github:abysssol/ollama-flake/1

Use an unchanging tagged version:

nix profile install github:abysssol/ollama-flake/1.6.0

Alternate packages can be specified as usual. From version 1, install the cpu package, which is built to only run on CPU:

nix profile install github:abysssol/ollama-flake/1#cpu

Other versions may be available:

nix profile install github:abysssol/ollama-flake/0
nix profile install github:abysssol/ollama-flake/1.1.0

Backend

Multiple packages are available for the different computation backends supported by ollama on linux. On other platforms (eg darwin), only the default package is available. On darwin, gpu acceleration via metal should work by default.

The available options:

cpu: fallback CPU implementation

nix profile install github:abysssol/ollama-flake#cpu

rocm: supported by modern AMD GPUs

nix profile install github:abysssol/ollama-flake#rocm

cuda: supported by modern NVIDIA GPUs; uses unfree licensed libraries
```
nix profile install github:abysssol/ollama-flake#cuda
```
gpu: build for both rocm and cuda, then dynamically load the relevant library at runtime
```
nix profile install github:abysssol/ollama-flake#gpu
```

The default is gpu:

# both of these are the default package, and are equivalent
nix profile install github:abysssol/ollama-flake
nix profile install github:abysssol/ollama-flake#default
# both of the above are equivalent to the one below
nix profile install github:abysssol/ollama-flake#gpu

Update

Find the index of the package to update:

nix profile list

Update the package at index:

nix profile upgrade $index

If nix is hesitant to download updates, force nix to download new files with --refresh:

nix profile upgrade --refresh $index

Remove

Find the index of the package to remove:

nix profile list

Remove the package at index:

nix profile remove $index

License

This software is dedicated to the public domain under the Creative Commons Zero. Read the CC0 in the LICENSE file or online.

Contribution

Any contribution submitted for inclusion in the project is subject to the CC0; that is, it is released into the public domain and all copyright to it is relinquished.

vrinek/ollama-flake