coetaur0/ESIM

Validation loss lower than training loss?

Lyn-bia opened this issue · 0 comments

Lyn-bia commented

Throughout my first 7 epochs, loss is always lower on validation set rather than training set. Did anything go wrong or the validation set is just too easy?

`* Training epoch 2:
Avg. batch proc. time: 0.0477s, loss: 0.4858: 100%|█████████████████████████████████████████████████████████| 17168/17168 [14:15<00:00, 20.07it/s]
-> Training time: 855.3887s, loss = 0.4858, accuracy: 81.1468%

  • Validation for epoch 2:
    -> Valid. time: 3.0217s, loss: 0.3845, accuracy: 85.4095%

  • Training epoch 3:
    Avg. batch proc. time: 0.0476s, loss: 0.4385: 100%|█████████████████████████████████████████████████████████| 17168/17168 [14:13<00:00, 20.12it/s]
    -> Training time: 853.2559s, loss = 0.4385, accuracy: 83.2263%

  • Validation for epoch 3:
    -> Valid. time: 2.9044s, loss: 0.3668, accuracy: 86.1613%

  • Training epoch 4:
    Avg. batch proc. time: 0.0477s, loss: 0.4120: 100%|█████████████████████████████████████████████████████████| 17168/17168 [14:14<00:00, 20.08it/s]
    -> Training time: 854.8605s, loss = 0.4120, accuracy: 84.4212%

  • Validation for epoch 4:
    -> Valid. time: 2.9331s, loss: 0.3626, accuracy: 86.4966%

  • Training epoch 5:
    Avg. batch proc. time: 0.0477s, loss: 0.3917: 100%|█████████████████████████████████████████████████████████| 17168/17168 [14:14<00:00, 20.08it/s]
    -> Training time: 854.8143s, loss = 0.3917, accuracy: 85.3156%

  • Validation for epoch 5:
    -> Valid. time: 2.9344s, loss: 0.3559, accuracy: 86.7608%

  • Training epoch 6:
    Avg. batch proc. time: 0.0476s, loss: 0.3766: 100%|█████████████████████████████████████████████████████████| 17168/17168 [14:14<00:00, 20.10it/s]
    -> Training time: 854.1151s, loss = 0.3766, accuracy: 85.9788%

  • Validation for epoch 6:
    -> Valid. time: 2.9510s, loss: 0.3426, accuracy: 87.2892%

  • Training epoch 7:
    Avg. batch proc. time: 0.0477s, loss: 0.3639: 100%|█████████████████████████████████████████████████████████| 17168/17168 [14:15<00:00, 20.08it/s]
    -> Training time: 855.0051s, loss = 0.3639, accuracy: 86.5372%

  • Validation for epoch 7:
    -> Valid. time: 2.9162s, loss: 0.3464, accuracy: 87.7058%`