Question regarding the validation process to pick the best model
Ze-Yang opened this issue · 1 comments
Ze-Yang commented
Hi, I find that the validation is done every epoch on the test set with only the current novel class annotations. However, it seems like the model will always be saved regardless of whether the validation mIoU gets better or not. That makes the validation totally in vain. Do you notice this problem? I think it was a legacy issue from MiB's code. Looking forward to your reply. Thanks.
arthurdouillard commented
I don't do early stopping.
You can use this option https://github.com/arthurdouillard/CVPR2021_PLOP/blob/main/argparser.py#L146 to have on validation and find the best hyperparameters.