This repository, llamafile-docker
, automates the process of checking for new releases of Mozilla-Ocho/llamafile
, building a Docker image with the latest version, and pushing it to Docker Hub.
You will have to download a pre-trained model using the gguf format. You can find some on hugging face. Please refer to the llamafile documentation for more information or report an issue if you need help.
- Docker
- A gguf pre-trained model
docker run -it --rm \
-v /path/to/gguf/model:/model \
iverly/llamafile-docker:main-latest -m /model -p {prompt}
You will see the output of the model in the terminal.
- Docker
- A gguf pre-trained model
docker run -it --rm \
-v /path/to/gguf/model:/model \
iverly/llamafile-docker:server-latest -m /model
The server will be listening on port 8080 and expose an ui to interact with the model.
Please refer to the llamafile documentation the available endpoints.
Contributions are welcome. Please follow the standard Git workflow - fork, branch, and pull request.
This project is licensed under the Apache 2.0 - see the LICENSE
file for details.