Reproduce the performance of ADE20K
Yi-Qi638 opened this issue · 7 comments
I am struggling to reproduce the ADE20K reported performance on 8K iters (just get the 45.64mIoU). Is there any suggestion about this? This is the config setting~
lr_config = dict(policy='poly', power=0.9, min_lr=1e-6, by_epoch=False,
warmup='linear',
warmup_iters=150,
warmup_ratio=1e-6)
optimizer = dict(type='AdamW', lr=0.001, weight_decay=0.0001,
paramwise_cfg=dict(custom_keys={'unet': dict(lr_mult=0.1),
'encoder_vq': dict(lr_mult=0.0),
'text_encoder': dict(lr_mult=0.0),
'norm': dict(decay_mult=0.)}))
Same here. I used the exact config provided in the repo and got a val mIoU of 49.3 instead of the 54.6 in the paper.
Hi, can you provide more details such as the global batch size, the number of iterations, and the versions of the libraries? I will also have another check of the released config.
Batch_size is 16
8 gpus on V100
8K iterations
lr_config = dict(policy='poly', power=0.9, min_lr=1e-6, by_epoch=False,
warmup='linear',
warmup_iters=150,
warmup_ratio=1e-6)
optimizer = dict(type='AdamW', lr=0.001, weight_decay=0.0001,
paramwise_cfg=dict(custom_keys={'unet': dict(lr_mult=0.1),
'encoder_vq': dict(lr_mult=0.0),
'text_encoder': dict(lr_mult=0.0),
'norm': dict(decay_mult=0.)}))
Hi, can you provide more details such as the global batch size, the number of iterations, and the versions of the libraries? I will also have another check of the released config.
Please follow the hyper-parameters in [here], which should be more stable to reproduce the results in our paper. I have successfully reproduced the results and here is the log, where you can also check the details environment and hyper-parameters.
I hope this would be helpful and please feel free to ask if there are still any questions~
Please follow the hyper-parameters in [here], which should be more stable to reproduce the results in our paper. I have successfully reproduced the results and here is the log, where you can also check the details environment and hyper-parameters.
I hope this would be helpful and please feel free to ask if there are still any questions~
Thanks for your kind reply~ but the report mIoU of multi-scale is 54.6, which is higher than the log file (53.6). Is there any different abut the model architecture or training details?
The mIoU in the log file is obtained by single-scale testing. You can run bash dist_test.sh <CONFIG_PATH> <CHECKPOINT_PATH> <NUM_GPUS> --eval mIoU --aug-test to compute the multi-scale mIoU.
I was able to reproduce it now using the correct effective batch size. Thank you!