Train Loss Oscillation

[Daisy007girl]

When I train the 2 stages, I find that the loss changes in a small oscillation from beginning to end even if it falls in a general. I think it is unrelated to the fix of learning rate for I just use the origin learing rate in the code and the loss keep falling in the overall trend. Why is the loss always jumping?

[solitarysandman]

Because you're training in batches? Please look up how mini-batch gradient descent works.
https://engmrk.com/mini-batch-gd/