Can u add more clarity about folders like lfw/meta/sizes, train,val,test and what are they used for?

[RUPESHKUMARPALLAI]

Can u add more clarity about folders like lfw/meta/sizes, train,val,test and what are they used for?

[sriktrako]

Hi @RUPESHKUMARPALLAI,
Are you able to convert the dataset in required format?

[RUPESHKUMARPALLAI]

Hi @sriktrako
What validation is doing is taking 2 images and checking whether 2 are the same or not. It requires images and a boolean array. You can skip the bcolz array in utils.py and make a NumPy array of images with normalization(that idea you can get from how images are passes in training). And the boolean array size is half of the number of images representing if pairs are the same or not. Now, these 2 will be your new lfw,lfw_issame array, By this, you can avoid meta datas.
Thanks

[sriktrako]

Hi @RUPESHKUMARPALLAI, thanks for replying. Appreciate your help.

My DATA_ROOT = 'D:/face.evoLVe.PyTorch/data/dataV1'
Actual data:
D:/face.evoLVe.PyTorch/data/dataV1/Id1/1.jpg 2.jpg .....
D:/face.evoLVe.PyTorch/data/dataV1/Id2/1.jpg 2.jpg .....
Don't have any files other than .jpg's inside dataV1 directory.

Here's the exact output when I run train.py:

Overall Configurations:
{'SEED': 1337, 'DATA_ROOT': 'D:/face.evoLVe.PyTorch/data/dataV1', 'MODEL_ROOT': './model', 'LOG_ROOT': './log', 'BACKBONE_RESUME_ROOT': './model/weights/backbone_ir50_asia.pth', 'HEAD_RESUME_ROOT': './', 'BACKBONE_NAME': 'IR_50', 'HEAD_NAME': 'ArcFace', 'LOSS_NAME': 'Focal', 'INPUT_SIZE': [112, 112], 'RGB_MEAN': [0.5, 0.5, 0.5], 'RGB_STD': [0.5, 0.5, 0.5], 'EMBEDDING_SIZE': 512, 'BATCH_SIZE': 512, 'DROP_LAST': True, 'LR': 0.1, 'NUM_EPOCH': 125, 'WEIGHT_DECAY': 0.0005, 'MOMENTUM': 0.9, 'STAGES': [35, 65, 95], 'DEVICE': device(type='cpu'), 'MULTI_GPU': True, 'GPU_ID': [0, 1], 'PIN_MEMORY': True, 'NUM_WORKERS': 0}

Number of Training Classes: 5749
Traceback (most recent call last):
File "train.py", line 84, in
lfw, cfp_ff, cfp_fp, agedb, calfw, cplfw, vgg2_fp, lfw_issame, cfp_ff_issame, cfp_fp_issame, agedb_issame, calfw_issame, cplfw_issame, vgg2_fp_issame = get_val_data(DATA_ROOT)
File "D:\srikarRnD\dev\face.evoLVe.PyTorch\util\utils.py", line 63, in get_val_data
lfw, lfw_issame = get_val_pair(data_path, 'lfw')
File "D:\srikarRnD\dev\face.evoLVe.PyTorch\util\utils.py", line 56, in get_val_pair
carray = bcolz.carray(rootdir = os.path.join(path, name), mode = 'r')
File "bcolz/carray_ext.pyx", line 1067, in bcolz.carray_ext.carray.cinit
File "bcolz/carray_ext.pyx", line 1369, in bcolz.carray_ext.carray._read_meta
FileNotFoundError: [Errno 2] No such file or directory: 'D:/face.evoLVe.PyTorch/data/dataV1\lfw\meta\sizes'

How to bypass this error? It's trying to get val data which requires meta, sizes