[Bug] No training log when the num of iterations smaller than default interval
Closed this issue · 2 comments
Tau-J commented
Prerequisite
- I have searched Issues and Discussions but cannot get the expected help.
- The bug has not been fixed in the latest version(https://github.com/open-mmlab/mmengine).
Environment
mmengine == 0.5.0
Reproduces the problem - code sample
Reproduces the problem - command or script
Reproduces the problem - error message
Additional information
When the number of iterations per epoch is smaller than the log interval, there's no training info be printed. Since important information like eta is in training logs, I suggest printing at least one line of training logs per epoch.
Here's an example of current training logs:
2023/01/31 13:44:41 - mmengine - INFO - Saving checkpoint at 200 epochs
2023/01/31 13:44:45 - mmengine - INFO - Evaluating MpiiPCKAccuracy (normalized by ``"head_size"``)...
2023/01/31 13:44:45 - mmengine - INFO - Epoch(val) [200][12/12] Head PCK: 96.958305 Shoulder PCK: 96.375085 Elbow PCK: 90.350029 Wrist PCK: 85.442129 Hip PCK: 89.800562 Knee PCK: 86.871427 Ankle PCK: 82.686027 PCK: 90.257132 PCK@0.1: 33.794920
2023/01/31 13:45:05 - mmengine - INFO - Exp name: rtmpose-m_finetune-aic-coco_8xb64-210e_mpii_20230131_123528
2023/01/31 13:45:35 - mmengine - INFO - Exp name: rtmpose-m_finetune-aic-coco_8xb64-210e_mpii_20230131_123528
2023/01/31 13:45:56 - mmengine - INFO - Exp name: rtmpose-m_finetune-aic-coco_8xb64-210e_mpii_20230131_123528
2023/01/31 13:46:17 - mmengine - INFO - Exp name: rtmpose-m_finetune-aic-coco_8xb64-210e_mpii_20230131_123528
2023/01/31 13:46:30 - mmengine - INFO - Exp name: rtmpose-m_finetune-aic-coco_8xb64-210e_mpii_20230131_123528
2023/01/31 13:46:37 - mmengine - INFO - Exp name: rtmpose-m_finetune-aic-coco_8xb64-210e_mpii_20230131_123528
2023/01/31 13:46:57 - mmengine - INFO - Exp name: rtmpose-m_finetune-aic-coco_8xb64-210e_mpii_20230131_123528
2023/01/31 13:47:18 - mmengine - INFO - Exp name: rtmpose-m_finetune-aic-coco_8xb64-210e_mpii_20230131_123528
2023/01/31 13:47:38 - mmengine - INFO - Exp name: rtmpose-m_finetune-aic-coco_8xb64-210e_mpii_20230131_123528
2023/01/31 13:47:58 - mmengine - INFO - Exp name: rtmpose-m_finetune-aic-coco_8xb64-210e_mpii_20230131_123528
2023/01/31 13:48:19 - mmengine - INFO - Exp name: rtmpose-m_finetune-aic-coco_8xb64-210e_mpii_20230131_123528
2023/01/31 13:48:19 - mmengine - INFO - Saving checkpoint at 210 epochs
2023/01/31 13:48:22 - mmengine - INFO - Evaluating MpiiPCKAccuracy (normalized by ``"head_size"``)...
HAOCHENYE commented
You can set ignore_last=False in the configuration of LoggerHook. Considering that the number of samples in the last iteration is uncertain and the averaged value could be relatively unstable, the log for the last iteration will be ignored by default.
Tau-J commented
Hi @HAOCHENYE , I mean, users need to know the eta, but for now, this kind of information will not be printed when the number of samples is small.