Shouldn't activation be after the batch norm layer?

[ag1988]

Apologies in advance; I am new to this stuff.

Thanks,
Ankit

[geifmany]

Both works, You are right that in the original bathnorm paper they put it before, but it is also works the other way.
Actually there is a debate what the right form, some people do before and some after.

[ag1988]

I see..interesting. I'll check the paper to see why this is the case.
Anyways, thanks for the prompt response!

Regards,
Ankit

[geifmany]

You are welcome. It is not the best source but you can read more about it here https://www.reddit.com/r/MachineLearning/comments/67gonq/d_batch_normalization_before_or_after_relu/ <https://www.reddit.com/r/MachineLearning/comments/67gonq/d_batch_normalization_before_or_after_relu/>

…

On 12 May 2018, at 19:38, Ankit Gupta ***@***.***> wrote: I see..interesting. I'll check the paper to see why this is the case. Anyways, thanks for the prompt response! Regards, Ankit — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#5 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AIRlOZIMcOEMDcD_wnRVGJbhh3II-3nYks5txxAKgaJpZM4T8dJk>.

[ag1988]

This is great. Thanks for your help Yonathan. On Sat, May 12, 2018, 7:44 PM Yonatan Geifman <notifications@github.com> wrote:

…

You are welcome. It is not the best source but you can read more about it here https://www.reddit.com/r/MachineLearning/comments/67gonq/d_batch_normalization_before_or_after_relu/ < https://www.reddit.com/r/MachineLearning/comments/67gonq/d_batch_normalization_before_or_after_relu/ > > On 12 May 2018, at 19:38, Ankit Gupta ***@***.***> wrote: > > I see..interesting. I'll check the paper to see why this is the case. > Anyways, thanks for the prompt response! > > Regards, > Ankit > > — > You are receiving this because you modified the open/close state. > Reply to this email directly, view it on GitHub < #5 (comment)>, or mute the thread < https://github.com/notifications/unsubscribe-auth/AIRlOZIMcOEMDcD_wnRVGJbhh3II-3nYks5txxAKgaJpZM4T8dJk >. > — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#5 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Age0GBihfZoZVr-ShvHYR2YMoopckIzfks5txxFpgaJpZM4T8dJk> .