Feature Normalization in the Signal Propagation Experiment

[ignasa007]

Hi, can I check what the logic is behind this line in the signal propagation experiment file? I imagine it's to do with the normalization,

$$\frac{h_u^{(m),f}}{\left\|h_u^{(m),f}\right\|}$$

as in the last equation on Page 20 of the paper. Firstly, the denominator of that term doesn't make sense to me since $h_u^{(m),f}$ is supposed to be a scalar. Is that a typo in the paper? Secondly, the implementation seems to suggest that we are summing all the feature values (over all nodes and all feature dimensions) to normalize, which isn't even a node-wise normalization constant. Moreover, the features from the source node are being used to compute the normalizer (which wouldn't be an issue if the normalization is instead supposed to be node-wise, since $d_G(v,v)=0$).

Not sure if this is the intended formulation, but I guess the aforementioned line can be updated to normalize by out.abs().sum(dim=1, keepdims=True), which would correspond to the summands

$$\frac{h_u^{(m),f}}{\left\|h_u^{(m)}\right\|_1}$$

If so, I can verify the implementation and create a PR.