bug: stable diffusion example failed to run
Closed this issue · 10 comments
xieydd commented
Description
I have try this demo , but looks like some image is not found.
envd file:
def build():
base(os="ubuntu20.04", language="python")
#config.pip_index(url = "https://pypi.tuna.tsinghua.edu.cn/simple")
install.python_packages([
"torch",
"transformers",
"diffusers",
])
The detail log :
v0.10.3: Pulling from moby/buildkit
59bf1c3509f3: Already exists
cebf54714d9e: Pull complete
0b2c85935cbb: Pull complete
bb8c60c7d403: Pull complete
Digest: sha256:0dc312b04eac1b44cd2cad566deb1e886c753109208affbbec8384f381ff7f38
Status: Downloaded newer image for moby/buildkit:v0.10.3
[+] ⌚ parse build.envd and download/cache dependencies 0.0s ✅ (finished)
[+] build envd environment 1173.6s (40/40) FINISHED
=> importing cache manifest from docker.io/tensorchord/python-cache:envd-v0.2.5-alpha.2 8.6s
=> docker-image://docker.io/library/ubuntu:20.04 8.6s
=> => resolve docker.io/library/ubuntu:20.04 8.6s
=> docker-image://docker.io/tensorchord/horust:v0.1.0 8.6s
=> => resolve docker.io/tensorchord/horust:v0.1.0 8.6s
=> docker-image://docker.io/tensorchord/envd-sshd-from-scratch:v0.2.5-alpha.2 8.6s
=> => resolve docker.io/tensorchord/envd-sshd-from-scratch:v0.2.5-alpha.2 8.6s
=> CACHED [internal] install built-in packages 0.0s
=> CACHED [internal] create conda directory 0.0s
=> CACHED [internal] install conda 0.0s
=> CACHED [internal] install horust 0.0s
=> CACHED mkdir /etc/horust/services 0.0s
=> CACHED mkdir /var/log/horust 0.0s
=> CACHED [internal] add envd-sshd from tensorchord/envd-sshd-from-scratch:v0.2.5-alpha.2 0.0s
=> [internal] create user group envd 135.7s
=> => sha256:bb612f7d1b41fcf22253a4579506d2f560d7f3e49952bada08227ff87763f252 639B / 639B 0.7s
=> => sha256:acafdb8c9ac9f6887a90766700c812dba00630a156355e2ad2477dbfe1cba25c 3.24MB / 3.24MB 9.8s
=> => sha256:27fc7881ac6c6b58ffc37af86a1911d367157218a0356050c858866e875acdff 147B / 147B 2.7s
=> => sha256:98e38b07830150d715c574c3f506f8e336c4f4b69ece5c406a2e1156afd0054f 143B / 143B 2.7s
=> => sha256:8632dca06c7933257e47b3177ca36ebf1c01eacabac84b8b04651ebdfd06dfa4 2.06MB / 2.06MB 6.1s
=> => sha256:98be348627b5625a20cd4b98ea55925c740e1b79d47a1ab94936690f6659b7c0 58.96MB / 58.96MB 107.0s
=> => sha256:b78a47138b912e3f989dbc33f472821fe284de16e7123754f4cbd5cbb6ba4f0c 121B / 121B 1.0s
=> => sha256:e7e41ebb053c87bf873ece53729851d731005a12373f118d0e540d3d5ab53240 94.35MB / 94.35MB 120.5s
=> => sha256:eaead16dc43bb8811d4ff450935d607f9ba4baffda4fc110cc402fa43f601d83 28.58MB / 28.58MB 62.8s
=> => extracting sha256:eaead16dc43bb8811d4ff450935d607f9ba4baffda4fc110cc402fa43f601d83 2.2s
=> => extracting sha256:e7e41ebb053c87bf873ece53729851d731005a12373f118d0e540d3d5ab53240 6.1s
=> => extracting sha256:b78a47138b912e3f989dbc33f472821fe284de16e7123754f4cbd5cbb6ba4f0c 0.0s
=> => extracting sha256:98be348627b5625a20cd4b98ea55925c740e1b79d47a1ab94936690f6659b7c0 4.9s
=> => extracting sha256:8632dca06c7933257e47b3177ca36ebf1c01eacabac84b8b04651ebdfd06dfa4 0.1s
=> => extracting sha256:98e38b07830150d715c574c3f506f8e336c4f4b69ece5c406a2e1156afd0054f 0.0s
=> => extracting sha256:27fc7881ac6c6b58ffc37af86a1911d367157218a0356050c858866e875acdff 0.0s
=> => extracting sha256:acafdb8c9ac9f6887a90766700c812dba00630a156355e2ad2477dbfe1cba25c 0.1s
=> => extracting sha256:bb612f7d1b41fcf22253a4579506d2f560d7f3e49952bada08227ff87763f252 0.0s
=> [internal] create user envd 1.7s
=> [internal] add user envd to sudoers 0.2s
=> [internal] mkdir config and cache dir 0.1s
=> [internal] initialize conda bash environment 0.4s
=> [internal] install system packages 0.0s
=> [internal] add conda environment to bashrc 0.2s
=> [internal] configure shell bash 0.0s
=> [internal] create conda environment: bash -c "/opt/conda/bin/conda create -n envd python=3.9" 45.0s
=> [internal] conda python environment 0.2s
=> pre-python stage 6.2s
=> => merging 6.2s
=> [internal] install conda packages 0.0s
=> [internal] create cache dir 0.2s
=> [internal] create dir for ssh key 0.1s
=> [internal] install ssh keys 0.0s
=> [internal] install ssh key 0.0s
=> [internal] setting pip cache mount permissions 0.0s
=> pip install torch transformers diffusers 598.9s
=> [internal] install PyPI packages 0.4s
=> [internal] generating the image 12.2s
=> => merging 12.2s
=> [internal] update alternative python to envd 0.4s
=> [internal] update alternative python3 to envd 0.1s
=> [internal] update alternative pip to envd 0.1s
=> [internal] update alternative pip3 to envd 0.1s
=> [internal] creating config dir 0.0s
=> [internal] setting prompt starship config 0.0s
=> [internal] setting prompt bash config 0.1s
=> [internal] create file /etc/horust/services/sshd.toml 0.0s
=> exporting to oci image format 362.5s
=> => exporting layers 162.7s
=> => exporting manifest sha256:263187a91c0b1de29cfa0abee2ddd0bae788aee227b28e699c94a8bd9f9024a8 0.0s
=> => exporting config sha256:aed4367652753b1b3b24a2c3b533a7dfe7f8a3191e84696d966f8b2267091be1 0.0s
=> => sending tarball 199.8s
error: No such image: stable-diffusion:dev
Reproduction
I jsut follow this demo , blocking in envd up.
Additional Info
- envd env
envd: v0.2.5-alpha.2
BuildDate: 2022-11-03T13:12:35Z
GitCommit: 46d24fd1331b1b0e223cb2d2223e17d648080bf1
GitTreeState: clean
GitTag: v0.2.5-alpha.2
GoVersion: go1.18.7
Compiler: gc
Platform: darwin/amd64
- dev env
Darwin CHRISYDXIE-MB2 20.6.0 Darwin Kernel Version 20.6.0: Wed Jun 23 00:26:31 PDT 2021; root:xnu-7195.141.2~5/RELEASE_X86_64 x86_64
active environment : envd
active env location : /Users/xieydd/opt/anaconda3/envs/envd
shell level : 2
user config file : /Users/xieydd/.condarc
populated config files : /Users/xieydd/.condarc
conda version : 22.9.0
conda-build version : 3.22.0
python version : 3.9.13.final.0
virtual packages : __osx=10.16=0
__unix=0=0
__archspec=1=x86_64
base environment : /Users/xieydd/opt/anaconda3 (writable)
conda av data dir : /Users/xieydd/opt/anaconda3/etc/conda
conda av metadata url : None
channel URLs : https://repo.anaconda.com/pkgs/main/osx-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/osx-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /Users/xieydd/opt/anaconda3/pkgs
/Users/xieydd/.conda/pkgs
envs directories : /Users/xieydd/opt/anaconda3/envs
/Users/xieydd/.conda/envs
platform : osx-64
user-agent : conda/22.9.0 requests/2.28.1 CPython/3.9.13 Darwin/20.6.0 OSX/10.16
UID:GID : 501:20
netrc file : None
offline mode : False
Message from the maintainers:
Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.
kemingy commented
I'm not able to reproduce this error on Linux. I guess it's related to the disk space. Can you check if Docker Desktop (or something else you're using) has enough disk spaces?
xieydd commented
Awesome, i found the storage resource limit of docker is 120 GB , and now 119.8 GB used.
Does we need pre check the resource? @kemingy
kemingy commented
Awesome, i found the storage resource limit of docker is 120 GB , and now 119.8 GB used.
Does we need pre check the resource? @kemingy
Yeah, we can do that and make the error message more friendly.
xieydd commented
@gaocegege
I think it is necessary to update the aigc blog, some code is not useful.
Can you reproduce it again?
gaocegege commented
@gaocegege I think it is necessary to update the aigc blog, some code is not useful. Can you reproduce it again?
Could you please show the err log?
xieydd commented
I have fixed it via this doc, need change image = pipe(prompt, guidance_scale=7.5)["sample"][0] to image = pipe(prompt, guidance_scale=7.5).images[0].
I just tested in T4 cuda env, must use fp16 mode, here is my env and code:
def build():
base(os="ubuntu20.04", language="python")
#config.pip_index(url = "https://pypi.tuna.tsinghua.edu.cn/simple")
install.cuda(version="11.2.2", cudnn="8")
install.python_packages([
"torch",
"transformers",
"diffusers",
])
import random
import sys
import os
from diffusers import StableDiffusionPipeline
import torch
from torch import autocast
def dummy(images, **kwargs):
return images, False
# Read prompt from command line
prompt = " ".join(sys.argv[1:])
model_id="CompVis/stable-diffusion-v1-4"
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16, revision="fp16", use_auth_token=os.environ['HUGGINGFACE_TOKEN'])
pipe.to("cuda")
pipe.safety_checker = dummy
# Run for 10 samples
for i in range(10):
n = random.randint(1000, 9999)
with autocast("cuda"):
image = pipe(prompt, guidance_scale=7.5).images[0]
image.save(f'{n}.jpeg')
if i==9:
exit(0)gaocegege commented
SGTM. The API is updated. Let's update https://github.com/tensorchord/envd-docs/blob/main/docs-zh/blog/stable-diffusion-cpu.md
gaocegege commented
Could you please help us fix the example? https://github.com/tensorchord/envd/blob/main/examples/stable-diffusion/main.py#L26
gaocegege commented
SGTM. The API is updated. Let's update https://github.com/tensorchord/envd-docs/blob/main/docs-zh/blog/stable-diffusion-cpu.md
The blog post is already updated.
xieydd commented
Could you please help us fix the example? https://github.com/tensorchord/envd/blob/main/examples/stable-diffusion/main.py#L26
I will fix it.