alibaba/EasyCV

Bug: 数据集目录相关

Opened this issue · 1 comments

zhangs669 commented

Thanks for your error report and we appreciate it a lot.

Checklist

  • I have searched related issues but cannot get the expected help.
  • The bug has not been fixed in the latest version.

Describe the bug
我在使用FastConvMAE在imagenet-1k数据集上做预训练的时候，报错为下图
2024-04-04_203659
我查看了报错的图片是存在的，不知道为什么路径报错后面会加上label，而且data_train_root无论怎么改报错的地址都不变就算为空也一样，而且就算我改变目录结构直接把图片放到imagenet文件夹而不是imagenet\train文件夹下时也报一样的错
To Reproduce

  • What command or script did you run?
    python -m torch.distributed.launch --nproc_per_node=3 --master_port=29930
    tools/train.py
    configs/selfsup/fast_convmae/fast_convmae_vit_base_patch16_8xb64_50e.py
    --work_dir ./work_dir
    --launcher pytorch
  • Did you make any modifications on the code or config? Did you understand what you have modified?
  • 我将EasyCV/configs/selfsup/fast_convmae/fast_convmae_vit_base_patch16_8xb64_50e.py中的修改如下：
  • data_train_list = 'data/imagenet/meta/train.txt'
  • data_train_root = 'data/imagenet/train'
  • What dataset did you use?
    imagenet-1k格式如下
    EasyCV/data/imagenet
    └── train
    └── n01440764
    └── n01443537
    └── ...
    └── val
    └── n01440764
    └── n01443537
    └── ...
    └── meta
    ├── train.txt
    ├── val.txt
    ├── ...

Environment

  • Please run python easycv/utils/collect_env.py to collect necessary environment information and paste it here.
  • sys.platform: linux
    Python: 3.7.16 (default, Jan 17 2023, 22:20:44) [GCC 11.2.0]
    CUDA available: True
    CUDA_HOME: /usr/local/cuda-11.7
    NVCC: Build cuda_11.7.r11.7/compiler.31442593_0
    GPU 0,1,2: NVIDIA Graphics Device
    GCC: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
    PyTorch: 1.8.0+cu111
    PyTorch compiling details: PyTorch built with:
    • GCC 7.3
    • C++ Version: 201402
    • Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
    • Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683)
    • OpenMP 201511 (a.k.a. OpenMP 4.5)
    • NNPACK is enabled
    • CPU capability usage: AVX2
    • CUDA Runtime 11.1
    • NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
    • CuDNN 8.0.5
    • Magma 2.5.2
    • Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,

TorchVision: 0.9.0+cu111
OpenCV: 4.9.0
MMCV: 1.4.4
EasyCV: 0.11.6

  • You may add addition that may be helpful for locating the problem, such as
    • How you installed PyTorch [e.g., pip, conda, source]
    • pip
    • Other environment variables that may be related (such as $PATH, $LD_LIBRARY_PATH, $PYTHONPATH, etc.)
Cathy0908 commented

mae的数据格式不需要lable，把每行的label去掉