IndexError: list index out of range
cobanov opened this issue · 2 comments
cobanov commented
I'm getting this error either huggingface or local training, I don't exactly what's the problem everything seems fine.
accelerate launch --config_file accelerate_local.yaml \
train_unconditional.py \
--dataset_name mertcobanov/audio-diffusion-256 \
--resolution 256 \
--output_dir ddpm-ema-audio-256 \
--num_epochs 100 \
--train_batch_size 2 \
--eval_batch_size 2 \
--gradient_accumulation_steps 8 \
--learning_rate 1e-4 \
--lr_warmup_steps 500 \
--mixed_precision no \
--push_to_hub True \
--hub_model_id audio-diffusion-256 \
--hub_token $(cat $HOME/.huggingface/token)
The following values were not passed to `accelerate launch` and had defaults used instead:
`--num_cpu_threads_per_process` was set to `12` to improve out-of-box performance
To avoid this warning pass in values for each of the problematic parameters or run `accelerate config`.
Downloading: 100%|██████████████████████████████████████████| 699/699 [00:00<00:00, 507kB/s]
Using custom data configuration mertcobanov--audio-diffusion-256-1545067e5255003f
Downloading and preparing dataset None/None (download: 9.98 MiB, generated: 9.98 MiB, post-processed: Unknown size, total: 19.96 MiB) to /home/mert/.cache/huggingface/datasets/mertcobanov___parquet/mertcobanov--audio-diffusion-256-1545067e5255003f/0.0.0/2a3b91fbd88a2c90d1dbbb32b460cf621d31bd5b05b934492fdef7d8d6f236ec...
Downloading data: 100%|████████████████████████████████| 10.5M/10.5M [00:00<00:00, 18.0MB/s]
Downloading data files: 100%|█████████████████████████████████| 1/1 [00:01<00:00, 1.89s/it]
Extracting data files: 100%|████████████████████████████████| 1/1 [00:00<00:00, 1764.54it/s]
Dataset parquet downloaded and prepared to /home/mert/.cache/huggingface/datasets/mertcobanov___parquet/mertcobanov--audio-diffusion-256-1545067e5255003f/0.0.0/2a3b91fbd88a2c90d1dbbb32b460cf621d31bd5b05b934492fdef7d8d6f236ec. Subsequent calls will reuse this data.
Cloning https://huggingface.co/mertcobanov/audio-diffusion-256 into local empty directory.
Epoch 0: 100%|█| 129/129 [00:38<00:00, 3.31it/s, ema_decay=0.974, loss=0.0125, lr=2.58e-5,
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1000/1000 [01:26<00:00, 11.53it/s]
Traceback (most recent call last):
File "/home/mert/development/audio-diffusion/train_unconditional.py", line 319, in <module>
main(args)
File "/home/mert/development/audio-diffusion/train_unconditional.py", line 241, in main
accelerator.trackers[0].writer.add_images(
IndexError: list index out of range
Waiting for the following commands to finish before shutting down: [[push command, status code: running, in progress. PID: 313172]].
Waiting for the following commands to finish before shutting down: [[push command, status code: running, in progress. PID: 313172]].
Waiting for the following commands to finish before shutting down: [[push command, status code: running, in progress. PID: 313172]].
Waiting for the following commands to finish before shutting down: [[push command, status code: running, in progress. PID: 313172]].
Traceback (most recent call last):
File "/home/mert/anaconda3/envs/audio-generation/bin/accelerate", line 10, in <module>
sys.exit(main())
File "/home/mert/anaconda3/envs/audio-generation/lib/python3.10/site-packages/accelerate/commands/accelerate_cli.py", line 43, in main
args.func(args)
File "/home/mert/anaconda3/envs/audio-generation/lib/python3.10/site-packages/accelerate/commands/launch.py", line 837, in launch_command
simple_launcher(args)
File "/home/mert/anaconda3/envs/audio-generation/lib/python3.10/site-packages/accelerate/commands/launch.py", line 354, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['/home/mert/anaconda3/envs/audio-generation/bin/python3.10', 'train_unconditional.py', '--dataset_name', 'mertcobanov/audio-diffusion-256', '--resolution', '256', '--output_dir', 'ddpm-ema-audio-256', '--num_epochs', '100', '--train_batch_size', '2', '--eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--learning_rate', '1e-4', '--lr_warmup_steps', '500', '--mixed_precision', 'no', '--push_to_hub', 'True', '--hub_model_id', 'audio-diffusion-256', '--hub_token', 'my_token']' returned non-zero exit status 1.
mattgallivan commented
I ran into the same issue. You'll need to install tensorboard (pip install tensorboard
) as it is missing from requirements.txt.
cobanov commented
Great, it is solved thank you @mattgallivan !