Training finishes in one minute??
Opened this issue · 1 comments
I try to train but it ends in less than a minute and there is nothing there,
G:\ComfyUI2\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI\custom_nodes\Lora-Training-in-Comfy/sd-scripts/train_network.py
The following values were not passed to accelerate launch and had defaults used instead:
--num_processes was set to a value of 0
--num_machines was set to a value of 1
--mixed_precision was set to a value of 'no'
--dynamo_backend was set to a value of 'no'
To avoid this warning pass in values for each of the problematic parameters or run accelerate config.
C:\Users\facil\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node(
WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for:
PyTorch 2.2.0+cu121 with CUDA 1201 (you have 2.2.0+cpu)
Python 3.10.11 (you have 3.10.11)
Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers)
Memory-efficient attention, SwiGLU, sparse and more won't be available.
Set XFORMERS_MORE_DETAILS=1 for more details
C:\Users\facil\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node(
prepare tokenizer
update token length: 225
Using DreamBooth method.
ignore directory without repeats / 繰り返し回数のないディレクトリを無視します: chiara
prepare images.
found directory G:\database\5_chiara contains 29 image files
No caption file found for 29 images. Training will continue without captions for these images. If class token exists, it will be used. / 29枚の画像にキャプションファイルが見つかりませんでした。これらの画像についてはキャプションなしで学習を 続行します。class tokenが存在する場合はそれを使います。
G:\database\5_chiara\20240109_192235000_iOS.png
G:\database\5_chiara\20240109_192301000_iOS.png
G:\database\5_chiara\20240109_192332000_iOS.png
G:\database\5_chiara\20240109_192344000_iOS.png
G:\database\5_chiara\20240109_192347000_iOS.png
G:\database\5_chiara\20240109_192351000_iOS.png... and 24 more
145 train images with repeating.
0 reg images.
no regularization images / 正則化画像が見つかりませんでした
[Dataset 0]
batch_size: 1
resolution: (512, 512)
enable_bucket: True
min_bucket_reso: 256
max_bucket_reso: 1584
bucket_reso_steps: 64
bucket_no_upscale: False
[Subset 0 of Dataset 0]
image_dir: "G:\database\5_chiara"
image_count: 29
num_repeats: 5
shuffle_caption: True
keep_tokens: 0
caption_dropout_rate: 0.0
caption_dropout_every_n_epoches: 0
caption_tag_dropout_rate: 0.0
caption_prefix: None
caption_suffix: None
color_aug: False
flip_aug: False
face_crop_aug_range: None
random_crop: False
token_warmup_min: 1,
token_warmup_step: 0,
is_reg: False
class_tokens: chiara
caption_extension: .txt
[Dataset 0]
loading image sizes.
100%|█████████████████████████████████████████████████████████████████████████████████| 29/29 [00:00<00:00, 160.17it/s]
make buckets
number of images (including repeats) / 各bucketの画像枚数（繰り返し回数を含む）
bucket 0: resolution (320, 704), count: 5
bucket 1: resolution (384, 640), count: 45
bucket 2: resolution (448, 576), count: 60
bucket 3: resolution (512, 512), count: 30
bucket 4: resolution (576, 448), count: 5
mean ar error (without repeats): 0.059714616310329205
preparing accelerator
loading model for process 0/1
load StableDiffusion checkpoint: G:\ComfyUI2\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI\models\checkpoints\epicrealism_naturalSinRC1VAE.safetensors
UNet2DConditionModel: 64, 8, 768, False, False
loading u-net:
loading vae:
loading text encoder:
Enable xformers for U-Net
Traceback (most recent call last):
File "G:\ComfyUI2\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI\custom_nodes\Lora-Training-in-Comfy\sd-scripts\train_network.py", line 1012, in
trainer.train(args)
File "G:\ComfyUI2\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI\custom_nodes\Lora-Training-in-Comfy\sd-scripts\train_network.py", line 236, in train
vae.set_use_memory_efficient_attention_xformers(args.xformers)
File "C:\Users\facil\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\diffusers\models\modeling_utils.py", line 255, in set_use_memory_efficient_attention_xformers
fn_recursive_set_mem_eff(module)
File "C:\Users\facil\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\diffusers\models\modeling_utils.py", line 251, in fn_recursive_set_mem_eff
fn_recursive_set_mem_eff(child)
File "C:\Users\facil\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\diffusers\models\modeling_utils.py", line 251, in fn_recursive_set_mem_eff
fn_recursive_set_mem_eff(child)
File "C:\Users\facil\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\diffusers\models\modeling_utils.py", line 251, in fn_recursive_set_mem_eff
fn_recursive_set_mem_eff(child)
File "C:\Users\facil\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\diffusers\models\modeling_utils.py", line 248, in fn_recursive_set_mem_eff
module.set_use_memory_efficient_attention_xformers(valid, attention_op)
File "C:\Users\facil\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\diffusers\models\attention_processor.py", line 255, in set_use_memory_efficient_attention_xformers
raise ValueError(
ValueError: torch.cuda.is_available() should be True but is False. xformers' memory efficient attention is only available for GPU
Traceback (most recent call last):
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "C:\Users\facil\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\accelerate\commands\launch.py", line 996, in
main()
File "C:\Users\facil\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\accelerate\commands\launch.py", line 992, in main
launch_command(args)
File "C:\Users\facil\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\accelerate\commands\launch.py", line 986, in launch_command
simple_launcher(args)
File "C:\Users\facil\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\accelerate\commands\launch.py", line 628, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['C:\Users\facil\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\python.exe', 'G:/ComfyUI2/ComfyUI_windows_portable_nvidia_cu121_or_cpu/ComfyUI/custom_nodes/Lora-Training-in-Comfy/sd-scripts/train_network.py', '--enable_bucket', '--pretrained_model_name_or_path=G:\ComfyUI2\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI\models\checkpoints\epicrealism_naturalSinRC1VAE.safetensors', '--train_data_dir=G:/database', '--output_dir=models/loras', '--logging_dir=./logs', '--log_prefix=chi', '--resolution=512,512', '--network_module=networks.lora', '--max_train_epochs=50', '--learning_rate=1e-4', '--unet_lr=1e-4', '--text_encoder_lr=1e-5', '--lr_scheduler=cosine_with_restarts', '--lr_warmup_steps=0', '--lr_scheduler_num_cycles=1', '--network_dim=32', '--network_alpha=32', '--output_name=chi', '--train_batch_size=1', '--save_every_n_epochs=10', '--mixed_precision=fp16', '--save_precision=fp16', '--seed=11', '--cache_latents', '--prior_loss_weight=1', '--max_token_length=225', '--caption_extension=.txt', '--save_model_as=safetensors', '--min_bucket_reso=256', '--max_bucket_reso=1584', '--keep_tokens=0', '--xformers', '--shuffle_caption', '--clip_skip=2', '--optimizer_type=AdamW8bit', '--persistent_data_loader_workers', '--log_with=tensorboard']' returned non-zero exit status 1.
Train finished
Prompt executed in 16.80 seconds
You've troubleshooted your Xformers and have given your images captioning, right?