wufeim/NeMo

OOD-CV data setup error

Closed this issue · 2 comments

Error

multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/miniconda3/envs/nemo2a/lib/python3.9/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/NeMo/data/prepare_ood_cv.py", line 174, in worker
    with open(os.path.join(ood_cv_pose_data_path, 'lists', 'TrainSet.txt'), 'r') as fp:
FileNotFoundError: [Errno 2] No such file or directory: '/home/NeMo/data/ood_cv/lists/TrainSet.txt'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/NeMo/data/prepare_ood_cv.py", line 270, in <module>
    main()
  File "/home/NeMo/data/prepare_ood_cv.py", line 266, in main
    prepare_ood_cv(cfg, args.workers)
  File "/home/NeMo/data/prepare_ood_cv.py", line 124, in prepare_ood_cv
    results = list(tqdm(pool.imap(worker, tasks), total=len(tasks)))
  File "/home/miniconda3/envs/nemo2a/lib/python3.9/site-packages/tqdm/std.py", line 1178, in __iter__
    for obj in iterable:
  File "/home/miniconda3/envs/nemo2a/lib/python3.9/multiprocessing/pool.py", line 870, in next
    raise value
FileNotFoundError: [Errno 2] No such file or directory: '/home/NeMo/data/ood_cv/lists/TrainSet.txt'

Command to reproduce -

python3 prepare_ood_cv.py  --config config/datasets/ood_cv.yaml

Similar error for validation data setup. There are no 'lists' downloaded for OOD-CV.

multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/miniconda3/envs/nemo2a/lib/python3.9/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/NeMo/data/prepare_ood_cv.py", line 191, in worker
    with open(os.path.join(ood_cv_pose_data_path, 'lists', f'{cate}_all.txt'), 'r') as fp:
FileNotFoundError: [Errno 2] No such file or directory: '/home/NeMo/data/ood_cv/lists/aeroplane_all.txt'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/NeMo/data/prepare_ood_cv.py", line 272, in <module>
    main()
  File "/home/NeMo/data/prepare_ood_cv.py", line 268, in main
    prepare_ood_cv(cfg, args.workers)
  File "/home/NeMo/data/prepare_ood_cv.py", line 125, in prepare_ood_cv
    results = list(tqdm(pool.imap(worker, tasks), total=len(tasks)))
  File "/home/miniconda3/envs/nemo2a/lib/python3.9/site-packages/tqdm/std.py", line 1178, in __iter__
    for obj in iterable:
  File "/home/miniconda3/envs/nemo2a/lib/python3.9/multiprocessing/pool.py", line 870, in next
    raise value
FileNotFoundError: [Errno 2] No such file or directory: '/home/NeMo/data/ood_cv/lists/aeroplane_all.txt'
wufeim commented

This commit 7634500 resolved the issue.