pre-train dataset

[FabienCode]

During the pre-training process, when I use the share-captioner_coco_lcs_sam_1246k_1107.json file, I receive an error indicating a missing dataset in SAM:
FileNotFoundError: [Errno 2] No such file or directory: '/data/hym/project/VLMs/MobileVLM/data/pretrain_data/sam/images/sa_411432.jpg'

Therefore, I attempted to switch the data_path to sharegpt4v_mix665k_cap23k_coco-ap9k_lcs3k_sam9k_div2k.json, but encountered another error:

Caught AssertionError in DataLoader worker process 0. Original Traceback (most recent call last): File "/home/hym/anaconda3/envs/mobilevlm/lib/python3.10/site-packages/torch/utils/data/_utils/worker.py", line 308, in _worker_loop data = fetcher.fetch(index) File "/home/hym/anaconda3/envs/mobilevlm/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 51, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/hym/anaconda3/envs/mobilevlm/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 51, in <listcomp> data = [self.dataset[idx] for idx in possibly_batched_index] File "/data/hym/project/VLMs/MobileVLM/mobilevlm/train/train.py", line 670, in __getitem__ data_dict = preprocess( File "/data/hym/project/VLMs/MobileVLM/mobilevlm/train/train.py", line 571, in preprocess return preprocess_plain(sources, tokenizer) File "/data/hym/project/VLMs/MobileVLM/mobilevlm/train/train.py", line 543, in preprocess_plain assert len(source) == 2 AssertionError File "/data/hym/project/VLMs/MobileVLM/mobilevlm/train/train.py", line 904, in train trainer.train() File "/data/hym/project/VLMs/MobileVLM/mobilevlm/train/train_mem.py", line 16, in <module> train() AssertionError: Caught AssertionError in DataLoader worker process 0. Original Traceback (most recent call last): File "/home/hym/anaconda3/envs/mobilevlm/lib/python3.10/site-packages/torch/utils/data/_utils/worker.py", line 308, in _worker_loop data = fetcher.fetch(index) File "/home/hym/anaconda3/envs/mobilevlm/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 51, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/hym/anaconda3/envs/mobilevlm/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 51, in <listcomp> data = [self.dataset[idx] for idx in possibly_batched_index] File "/data/hym/project/VLMs/MobileVLM/mobilevlm/train/train.py", line 670, in __getitem__ data_dict = preprocess( File "/data/hym/project/VLMs/MobileVLM/mobilevlm/train/train.py", line 571, in preprocess return preprocess_plain(sources, tokenizer) File "/data/hym/project/VLMs/MobileVLM/mobilevlm/train/train.py", line 543, in preprocess_plain assert len(source) == 2 AssertionError
What should I do to resolve this issue?
Are there some data sets I'm missing, can you provide some solutions or provide a complete pre-train Data Structure?