/SnakeActivationFunction

code for the paper "Neural Networks Fail to Learn Periodic Functions and How to Fix It"

Primary LanguageJupyter Notebook

code for the paper "Neural Networks Fail to Learn Periodic Functions and How to Fix It". This wouldve saved me a lot of time xD

TODO:

  1. try initialization schemes mentioned in the paper
  2. how to prevent nans
  3. two params (done)
  4. why isnt snake more common?
  5. compare against relu and co (some experiments in the paper left this out)
  6. fastai version?