IndicoDataSolutions/Passage

Implement learning rate schedule

ma2rten opened this issue · 1 comments

I was able to get better results with SGD on my own dataset with this library, if I use SGD and half the learning rate after each epoch, than using Eddy without such a learning rate schedule.

It would be nice if this was an official feature. I would make a pull request, but I am not sure what the best way is to implement it.

Mind posting your unofficial working code? I would be very excited to see what you have, and perhaps, it would give us a starting point for making it an official feature.

There's definitely already a great selection of update rules in Passage, I'm wondering if you've tried any of them for your task in substitution for scheduled-halving-SGD