Can we have a docker installation option?
ye7iaserag opened this issue · 13 comments
This is just a suggestion, having docker container support would make things way easier to install and isolate, also with docker compose you can start everything including gpu support with a simple docker-compose up
conda is contained, and so is micromamba (for the 1-click installers).
What niche would docker serve?
A one command to run on all platforms and a guarantee that it works
Btw I've already created the Dockerfile if anyone needs it
FROM python:3.10.6-slim-bullseye
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y git software-properties-common gnupg
COPY . /app
WORKDIR /app
RUN dpkg -i /app/installers/cuda-repo-debian11-11-7-local_11.7.1-515.65.01-1_amd64.deb
RUN cp /var/cuda-repo-debian11-11-7-local/cuda-*-keyring.gpg /usr/share/keyrings/
RUN add-apt-repository contrib
RUN apt-get update
RUN apt-get -y install cuda \
&& apt -y remove nvidia-* \
&& rm -rf /var/cuda-repo-debian11-11-6-local
RUN --mount=type=cache,target=/root/.cache/pip pip install -r /app/requirements.txt
RUN --mount=type=cache,target=/root/.cache/pip pip install -r /app/extensions/google_translate/requirements.txt
RUN --mount=type=cache,target=/root/.cache/pip pip install -r /app/extensions/silero_tts/requirements.txt
CMD python server.py --auto-devices --cai-chat --load-in-8bit --bf16 --listen --listen-port=8888
I also created the following docker-compose.yml
version: "3.3" # optional since v1.27.0
services:
text-generation-webui:
build: .
ports:
- "8889:8888"
stdin_open: true
tty: true
volumes:
- .:/app
deploy:
resources:
reservations:
devices:
- driver: nvidia
device_ids: ['0']
capabilities: [gpu]
and had to add those dependencies to the bottom of the requirements.txt
--extra-index-url https://download.pytorch.org/whl/cu117
torchaudio
torch==1.13.1+cu117
torchvision==0.14.1+cu117
feel free to close the PR, and for anyone coming from search, you're welcome :D
Thanks for posting the Dockerfile, @ye7iaserag!
Thanks, ye7iaserag!
And for why docker, personally I avoid installing conda on my machine.
It's messed up python before on my machine, so I avoid it if I can. Having everything existing in docker makes that a simple workaround.
Just to add to the topic, for stable-diffusion there is a docker-focused project in AbdBarho/stable-diffusion-webui-docker.
Another benefit is that Docker may potentially be an extra layer of security in case you're adding third-party dependencies/models/extensions, and in my opinion it's the best way to organize dependencies and such.
I always have an updated docker if anyone needs it just msg me
@ye7iaserag - First, thanks for your efforts here, with all of the back and forth trouble this repo has been giving me, a solution like Docker seems ideal. With that said, please forgive my ignorance as Docker is brand new to me but I'm trying to make sense of it. :)
I installed Docker in Win 10, created an empty directory, made a Dockerfile and docker-compose.yml file with the contents you supplied and placed them both in the folder. Then unzipped the oobabooga repo into the same folder and updated the requirements.txt files with the additions you listed.
I then ran docker build . first as I was going off of the Docker website documentation on Dockerfiles and it started doing a lot of stuff and then errored out with this error:
> [stage-0 5/12] RUN dpkg -i /app/installers/cuda-repo-debian11-11-7-local_11.7.1-515.65.01-1_amd64.deb: #0 0.422 dpkg: error: cannot access archive '/app/installers/cuda-repo-debian11-11-7-local_11.7.1-515.65.01-1_amd64.deb': No such file or directory
I then re-read your first message and realized I maybe should have used docker-compose up instead, so I ran that and it, too, errored out with the same error.
Is this just saying I need to get a copy of that debian distro and place it in the folder as well? Or, am I completely off base with the whole thing?
Is this just saying I need to get a copy of that debian distro and place it in the folder as well?
For anyone else new to Docker, this appears to be the answer. I downloaded that distro from nvidia and placed a copy in my folder in an installers subfolder and then re-ran docker-compose up and it built the image then started up the container.
Unfortunately, now when I try to access the ui using `localhost:8889', I get a "This page isn't working" error in the browser. So, the container is running, but something is still wrong. No errors reported in the Docker container logs. Just has the following output:
2023-03-18 04:49:29 Python 3.10.6 (main, Aug 23 2022, 08:36:38) [GCC 10.2.1 20210110] on linux
2023-03-18 04:49:29 Type "help", "copyright", "credits" or "license" for more information.
EDIT: Looks like I have to run the python server.py --auto-devices --cai-chat --load-in-8bit --bf16 --listen --listen-port=8888 command manually in the terminal after starting the container. Once I do that, the ui starts up and I can access it in the browser via localhost:8889 as expected. I assumed since it was in the CMD line at the end of the Dockerfile that this would happen automatically. I could still be missing something, but at least it appears to be working. :)
This cuda-repo-debian11-11-7-local_11.7.1-515.65.01-1_amd64.deb is the cuda local repo, you need it if you want to run with GPU, and you can get it on nvidia developer site.
here: https://developer.download.nvidia.com/compute/cuda/11.7.1/local_installers/cuda-repo-debian11-11-7-local_11.7.1-515.65.01-1_amd64.deb
Put it under installers folder, then docker-compose up --build to rebuild the image and install cuda
Also I've posted an updated version here #373
Also I've posted an updated version here #373
Thank you so much for your help and the updates!
does it work for mac users?
@BaptisteGarcin docker works anywhere and everywhere, that's one of its design principles/goals, here is a link for mac https://docs.docker.com/desktop/install/mac-install/
Also you do not need to use my implementation, a docker implementation was committed to the repo a couple of days ago and should be more up to date