/ollama-compose

Ollama docker compose.

Primary LanguageYAMLMIT LicenseMIT

README

Ollama docker compose.

Prerequisites

Hardware Requirements

Nvidia GPU.

Memory Requirements

  • 7b models generally require at least 8GB of RAM
  • 13b models generally require at least 16GB of RAM
  • 70b models generally require at least 64GB of RAM

Getting Started

docker compose up -d
docker exec -it ollama bash
ollama run llama2

References

  • Ollama provides a REST API for running and managing models.
  • Ollama supports a list of open-source models available on ollama.ai/library.