code for the paper "Neural Networks Fail to Learn Periodic Functions and How to Fix It". This wouldve saved me a lot of time xD
TODO:
- try initialization schemes mentioned in the paper
- how to prevent nans
- two params (done)
- why isnt snake more common?
- compare against relu and co (some experiments in the paper left this out)
- fastai version?