/Find-Best-Learning-Rate-TensorFlow

As the name, a method to find best learning rate.

Primary LanguagePython

Find-Best-Learning-Rate

This repo aims to find the best learning rate for my deep learning research.

Usage

Download the full repo and just run:

$ python3 template.py

It is an example for MNIST data, the lr_schedule and Loss changes can be found in the two jpg files.

Lr schedule:

lr_schedule

MNIST Loss changes:

output

Face Recognition

Src Repo: https://github.com/Ecohnoch/Tiny-Face-Recognition

Set lr schedule:

lr_schedule

Loss changes:

output

So we can find 1e-4 is the best original learning rate for my face recognition.

To be continued.

Reference

 Smith L N . Cyclical Learning Rates for Training Neural Networks[J]. Computer Science, 2015:464-472.