Cyril9227/Keras_AttentiveNormalization

3D AN?

Closed this issue · 2 comments

OverLordGoldDragon commented

It appears this implementation only supports 4D inputs - though Conv1D takes and outputs 3D tensors; I found that making below changes enables AN to work for 3D tensors without errors:

GlobalAveragePooling2D() --> GlobalAveragePooling1D()
gamma_readjust[:, None, None, :] --> gamma_readjust[:, None, :]
beta_readjust[:, None, None, :] --> beta_readjust[:, None, :]

I'm unsure, however, whether this is a correct implementation consistent with the paper - can you confirm? Thanks

Cyril9227 commented

Thank's for your feedback.

Sure, you are right. Feel free to make a pull request for this

  • 👍1
OverLordGoldDragon commented

Thank you for your implementation