run inference with released weights

[zhangai6666]

Hi Folks,

Thank you so much for releasing the weights! I'm having difficulty running with it. Based on my understanding of the paper, the input to the inference has:(marked in the pictures below)
(1) Original image, which generates the 18 critical body points for conditioning
(2) Garment Sketch for conditioning
(3) Text prompt for conditioning

However, --dataset is a required argument for eval.py. Does it mean we need the entire training dataset(thousands of garment pictures and captions) in order to run inference?
When I followed the README to run the following command. It errored out:

python eval.py --dataset_path .\assets\data\dresscode --batch_size 1 --mixed_precision fp16 --output_dir .\output\ --save_name test1  --num_workers_test 1 --sketch_cond_rate 0.2 --dataset dresscode --start_cond_rate 0.0
Using cache found in C:\Users\denni/.cache\torch\hub\aimagelab_multimodal-garment-designer_main
  0%|                                                                                                                                                                      | 0/4 [00:01<?, ?it/s]
Traceback (most recent call last):
  File "eval.py", line 157, in <module>
    main()
  File "eval.py", line 152, in main
    seed = args.seed,
  File "C:\Users\denni\miniconda3\envs\mgd\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\denni\Source\Repos\multimodal-garment-designer\utils\image_from_pipe.py", line 58, in generate_images_from_inpaint_sketch_posemap_pipe
    for batch in tqdm(test_dataloader):
  File "C:\Users\denni\miniconda3\envs\mgd\lib\site-packages\tqdm\std.py", line 1178, in __iter__
    for obj in iterable:
  File "C:\Users\denni\miniconda3\envs\mgd\lib\site-packages\accelerate\data_loader.py", line 377, in __iter__
    current_batch = next(dataloader_iter)
  File "C:\Users\denni\miniconda3\envs\mgd\lib\site-packages\torch\utils\data\dataloader.py", line 681, in __next__
    data = self._next_data()
  File "C:\Users\denni\miniconda3\envs\mgd\lib\site-packages\torch\utils\data\dataloader.py", line 1376, in _next_data
    return self._process_data(data)
  File "C:\Users\denni\miniconda3\envs\mgd\lib\site-packages\torch\utils\data\dataloader.py", line 1402, in _process_data
    data.reraise()
  File "C:\Users\denni\miniconda3\envs\mgd\lib\site-packages\torch\_utils.py", line 461, in reraise
    raise exception
NotImplementedError: Caught NotImplementedError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "C:\Users\denni\miniconda3\envs\mgd\lib\site-packages\torch\utils\data\_utils\worker.py", line 302, in _worker_loop
    data = fetcher.fetch(index)
  File "C:\Users\denni\miniconda3\envs\mgd\lib\site-packages\torch\utils\data\_utils\fetch.py", line 49, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "C:\Users\denni\miniconda3\envs\mgd\lib\site-packages\torch\utils\data\_utils\fetch.py", line 49, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "C:\Users\denni\Source\Repos\multimodal-garment-designer\datasets\dresscode.py", line 247, in __getitem__
    raise NotImplementedError
NotImplementedError

I would greatly appreciate any help from the contributors and other folks. It would be great if someone can create a dedicated inference.py with out-of-the-box working inference example.

[omedivad]

Hello @Dennisboyangzhang,

It appears that there may be an issue with the --dataset_path parameter you provided. Would you please try using an absolute path as the parameter of --dataset_path and let me know if the issue persists?

[zhangai6666]

Hi @omedivad ,

Really appreciate your reply. I was able to run the inference following your comment. The default command with default input files generated the result in the picture attached . It has some unexpected distortion on the body, especially the missing arms. Maybe I'm missing some setting or inputs, so I double checked that I’m using all expected inputs from the repo and dataset.

I would greatly appreciate it if you or other folks could help take a look. Let me know if you would like me to create a debug repo. Thank you so much!

[zhangtao22]

@Dennisboyangzhang hi,how could you get the dresscode dataset?Thank you!

[zhangai6666]

@zhangtao22 Hi, you might want to sign the data request form here:
https://github.com/aimagelab/dress-code