ValueError: num_samples should be a positive integer value, but got num_samples=0

[teytaud]

This looks like a simple thing like an argument missing but if you have a suggestion I'm interested.
The command line was << bash scripts/train.sh yc2 true >>.

Traceback (most recent call last):
File "src/train.py", line 706, in
main()
File "src/train.py", line 642, in main
num_workers=opt.num_workers, pin_memory=opt.pin_memory)
File "XXX/.conda/envs/videocap/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 270, in init
sampler = RandomSampler(dataset, generator=generator) # type: ignore[arg-type]
File "XXX.conda/envs/videocap/lib/python3.7/site-packages/torch/utils/data/sampler.py", line 103, in init
"value, but got num_samples={}".format(self.num_samples))
ValueError: num_samples should be a positive integer value, but got num_samples=0

[chunhuizhang]

please check your Dataset, which may empty (len == 0) causes the error.

[teytaud]

du -ks data/anet/*
16075976 data/anet/anet_c3d
14476240 data/anet/anet_c3d.zip
2404464 data/anet/anet_clip_b16
505216 data/anet/anet_clip_b16.zip

[teytaud]

Maybe the commands in the readme prepare anet and not yc2 ?
Thanks for your help.

[Kashu7100]

The provided features are for the ActivityNet. You need to extract the features for YouCook2 in order to run with the yc2 option.
Please refer to #2 for the feature extraction.

[JiangTianyu51147]

I have the same error when I try to train MART model on ActivityNet Captions,but i don't have any way to fix it.What shall I do?If you can give some suggestions I will sincerely thank you！

[Kashu7100]

@JiangTianyu51147 Could you tell me more about your situation (what command you run, which dataset you tried, etc.) so that I can help you?

[JiangTianyu51147]

I input the command

bash scripts/train.sh anet true

and reported an error

Traceback (most recent call last):
File "src/train.py", line 684, in
main()
File "src/train.py", line 632, in main
num_workers=opt.num_workers, pin_memory=opt.pin_memory)
File "/XXX/anaconda3/envs/vlcap/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 270, in init
sampler = RandomSampler(dataset, generator=generator) # type: ignore[arg-type]
File "/XXX/anaconda3/envs/vlcap/lib/python3.7/site-packages/torch/utils/data/sampler.py", line 103, in init
"value, but got num_samples={}".format(self.num_samples))
ValueError: num_samples should be a positive integer value, but got num_samples=0

I have downloaded env feature and lang feature and my VLCAP folder directory is as follows:

|── cache
|── data
│   └── anet
│   ├── env_c3d
│   ├── lang_feature
│   └── sent_feature
├── densevid_eval
├── scripts
├── src
│   └── rtransformer
└── video_feature

and I have changed data_path in VLCAP/src/rtransformer/recursive_caption_dataset.py but it still reports the same error.
I don't know what should I do.Thank you very much for your help! :)

[Kashu7100]

Can I know what is your current setting in RecursiveCaptionDataset?
num_samples=0 error typically occurs when you have wrong path.

data_path = {
        "anet": {
            "env_feat": "data/anet/anet_c3d/",
            "clip_feat": "data/anet/anet_clip_b16/lang_feature/",
            "sent_feat": "data/anet/anet_clip_b16/sent_feature/"
        },
        "yc2": {
            "env_feat": "data/yc2/yc2_c3d/",
            "clip_feat": "data/yc2/yc2_clip_b16/lang_feature/",
            "sent_feat": "data/yc2/yc2_clip_b16/sent_feature/"
        }
    }

[JiangTianyu51147]

This is my setting in XXX/VLCAP/src/rtransformer/recursive_caption_dataset.py

data_path = {
        "anet": {
            "env_feat": "XXX/VLCAP/data/anet/env_c3d/",
            "clip_feat": "XXX/VLCAP/data/anet/anet_clip_b16/lang_feature/",
            "sent_feat": "XXX/VLCAP/data/anet/anet_clip_b16/sent_feature/"
        },
        "yc2": {
            "env_feat": "XXX/VLCAP/data/yc2/yc2_c3d/",
            "clip_feat": "XXX/VLCAP/data/yc2/yc2_clip_b16/lang_feature/",
            "sent_feat": "XXX/VLCAP/data/yc2/yc2_clip_b16/sent_feature/"
        }
    }

and now my VLCAP folder directory is as follows:
├── cache
├── data
│   └── anet
│   ├── anet_clip_b16
│   │   ├── lang_feature
│   │   └── sent_feature
│   └── env_c3d
├── densevid_eval
│   ├── anet_data
│   ├── coco-caption
│   │   ├── annotations
│   │   ├── pycocoevalcap
│   │   │   ├── bleu
│   │   │   │   └── pycache
│   │   │   ├── cider
│   │   │   │   └── pycache
│   │   │   ├── meteor
│   │   │   │   ├── data
│   │   │   │   └── pycache
│   │   │   ├── pycache
│   │   │   ├── rouge
│   │   │   │   └── pycache
│   │   │   └── tokenizer
│   │   │   └── pycache
│   │   └── pycocotools
│   └── yc2_data
├── results
├── scripts
├── src
│   ├── pycache
│   └── rtransformer
│   └── pycache
└── video_feature
Thank you! :)

[Kashu7100]

Unless it is a full path, maybe you don't need "XXX/VLCAP/" in your path.

[JiangTianyu51147]

This error is finally solved and it really is a path error.I changed the path and it is running now.Once again sincerely thank you! :):):)