Loss only decrease from 2.2 to 1.8

Question

Loss only decrease from 2.2 to 1.8

yqNLP opened this issue 5 years ago · 2 comments

I use the default setting and your dataset, running 2w batches. It seems the loss wouldn't decease too much, only from 2.2 to 1.8. I wonder if there is something wrong with my experiments?

Answer 1 · 2019-07-28T07:15:10.000Z

the above problem is occurred in first-order, second-order wouldn't have that problem.

Answer 2 · 2020-06-28T15:45:13.000Z

I have the same problem. The loss is not decreasing.