piEsposito/blitz-bayesian-deep-learning

Bayesian representation of non-linear layers

cappelletto opened this issue · 2 comments

cappelletto commented

Are there any plans to expand the set of Bayesian implementations for non-linear layers (e.g. sigmoid, tanh)? Or they already exist, but maybe I'm just failing to find them. I am more than glad to provide some time/effort towards the inclusion of this feature.

Cheers,
Jose
PS: Thanks for this library, I've tested 5+ different existing solutions, and Blitz is perfect for my goals of rapid prototyping and testing.

piEsposito commented

Hello @cappelletto, and thank you for your kind words.

These non-linear layers can be built by using stacking a Bayesian Layer and a non-linear activation. As BLiTZ is fully integrated with PyTorch, you can create the layers and, on the forward method of the nn.Module you put your on-linear activations.

Hope that answers your question.
Best regards,
Pi

  • 👍1
cappelletto commented

Hi @piEsposito

Thanks for your reply. Stacking linear Bayesian layers with non-Bayesian non-linear activation functions will do the trick. Once again, thanks for the library.

Bests,
Jose

  • 👍1