/imagenet_prepare

Extract imagenet dataset and convert to .npy format(optional)

Primary LanguagePython

Prepare ImageNet-1K

To extract ImageNet dataset as following structure:

  train/
  ├── n01440764
  │   ├── n01440764_10026.JPEG
  │   ├── n01440764_10027.JPEG
  │   ├── ......
  ├── ......
  val/
  ├── n01440764
  │   ├── ILSVRC2012_val_00000293.JPEG
  │   ├── ILSVRC2012_val_00002138.JPEG
  │   ├── ......
  ├── ......

1. Download from http://www.image-net.org/challenges/LSVRC/2012/downloads

(Needed to login and you can see the download link)

2. For training dataset

mkdir train && mv ILSVRC2012_img_train.tar train/ && cd train
tar -xvf ILSVRC2012_img_train.tar
find . -name "*.tar" | while read NAME ; do mkdir -p "${NAME%.tar}"; tar -xvf "${NAME}" -C "${NAME%.tar}"; rm -f "${NAME}"; done
mv ILSVRC2012_img_train.tar ../
cd ..

3. For validating dataset

mkdir val && mv ILSVRC2012_img_val.tar val/ && cd val && tar -xvf ILSVRC2012_img_val.tar
mv ILSVRC2012_img_val.tar ../
wget -qO- https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/valprep.sh | bash
cd ..

or

mkdir val && mv ILSVRC2012_img_val.tar val/ && cd val && tar -xvf ILSVRC2012_img_val.tar
wget -qO- https://files-cdn.cnblogs.com/files/luruiyuan/valprep.sh | bash
cd ..

4. For test dataset

(Don't need to make test directory, the test tar file contains it.)

tar -xvf ILSVRC2012_img_test_v10102019.tar

Convert imagenet to .npy format(Optional)

python imagenet2npy.py --src /your/path/to/imagenet --dst /your/path/to/imageet_npy --workers 4 --resume

Convert imagenet to lmdb format(Optional)

python imagenet2lmdb.py -f /your/path/to/imagenet -s train -p 8
python imagenet2lmdb.py -f /your/path/to/imagenet -s val -p 8