spatial information implicitly used in MNIST example

[chentingpc]

Hi, thanks for opensource!

I found that in your MNIST example, spatial information (or node ordering), is used implicitly. This is due to that a set of nodes are treated as a list and fed into fully contented layer (in here, [N, M, F] reshaped into [N, M*F]). This seems to implicitly use something more than just the grid graph, as for graph this set of nodes should be treated in a permutation-invariant way. A consequence seems to be that during test time, if the input is rotated 90 degree, the model will fail since data are not truly treated as graph.

Could you please kindly confirm this?

[mdeff]

That's right: You need a permutation-invariant function if you want true invariance. People often use global average pooling (GAP).