yu4u/age-gender-estimation

Epoch 00001: val_loss did not improve from inf

Opened this issue · 7 comments

yimjunhyuck2 commented

I tried training through the following command but failed : sudo python3 ./train.py --input data/imdb_db.mat --nb_epochs 10 --depth 10. Every epoch went on with message "Epoch 000XX: val_loss did not improve from inf". Also after train.py was done, I tried plotting by "sudo python3 ./plot.history.py" but it also failed with a message "File models/history_16_8.h5 does not exist". I wonder if it's because I'm missing something.

yimjunhyuck2 commented

During training, the value of loss become nan. I can't find how to solve this problem.

yu4u commented

How did you create data/imdb_db.ma?

Try different learning rate lr.

yimjunhyuck2 commented

I created data/imdb_db.mat and tried lr 0.6, 0.8, 1.5. But the nan loss problem still occur. :(

yimjunhyuck2 commented

Is this a problem with tensorflow or keras?

youngmihuang commented

I want to know if there's solution from it? I have the same problem here.

yimjunhyuck2 commented
  • 👍1
youngmihuang commented

Thank for your suggestion. It seems that decreasing the learning rate lower than 0.01, and the loss would be improved (not NaN)?