seongjunyun/Graph_Transformer_Networks

GTConv Weight Initialization

supfisher opened this issue · 1 comments

In the class "GTConv", I saw that you initialize "self.weight" by nn.init.constant_(self.weight, 0.1).
I don't really understand why the model weights are initialized constantly.
Is there any specific reason for you?

Hi, sorry for the late reply.
That's a good question. we initialize weights of all edge types equally to prevent the convex combination of adjacency matrices from being biased to a specific edge type at the beginning of training.
But since we found that the problem that we were worried about did not occur even with the random initialization, so I change it to the random initialization.