huggingface/pytorch-image-models

i want to load data not only .tar file but also .tar.gz

TheDarkKnight-21th opened this issue · 1 comments

TheDarkKnight-21th commented

Describe the bug
A clear and concise description of what the bug is.

Now, timm library has just load .tar dataset. but i wanna load .tar.gz.

( my dataset is imagenet21k winter which is winter21_whole.tar.gz)

what should i do?

To Reproduce
Steps to reproduce the behavior:

  1. just download the imagenet21k winter
  2. run the timm/train.py

Expected behavior
A clear and concise description of what you expected to happen.
maybe there is error because timm can load tar file only

Screenshots
If applicable, add screenshots to help explain your problem.

plz check the python file (/timm/data/readers/reader_image_in_tar.py)

Desktop (please complete the following information):

  • OS: [e.g. Windows 10, Ubuntu 18.04]
  • This repository version [e.g. pip 0.3.1 or commit ref]
  • PyTorch version w/ CUDA/cuDNN [e.g. from conda list, 1.7.0 py3.8_cuda11.0.221_cudnn8.0.3_0]

Additional context
Add any other context about the problem here.

rwightman commented

@TheDarkKnight-21th that won't work, you have to at least un-gzip to .tar ... cannot random access within gz