dmbernaal/Daedalus

Deep Learning Research

Deep Learning Research

R1: Normalizing Activation with Good Init and Activations

In this notebook we will explore different initialization methods along with activation function and see how each effect the activation output with $N$ layers.

Links to papers:

• https://arxiv.org/ftp/arxiv/papers/1908/1908.08681.pdf
• https://arxiv.org/pdf/1511.06422.pdf
• https://arxiv.org/pdf/1502.01852.pdf

R2: Self Normalising Neural Networks

In this notebook we will explore Self-Normalising-Neural-Networks

Links to papers:

• https://medium.com/@damoncivin/self-normalising-neural-networks-snn-2a972c1d421
• https://becominghuman.ai/paper-repro-self-normalizing-neural-networks-84d7df676902
• https://arxiv.org/pdf/1706.02515.pdf