. A quick processing-sketch to compare the speed of different activation functions for the Processing PDE.
###results. average results for my machine for 1000000 passes throgh the activation function + calculating it's gradient:
Sigmoid: 1950 ms SoftSign: 4 ms Tanh: 3940 ms Rectified Linear: 3 ms ArcTan 475 ms