Reasoning behind `convert_to_single_emb`

[sidnb13]

I am trying to apply TokenGT to the 2D_data_npj molecular dataset for property prediction. I am struggling to understand why the following function is applied during the preprocessing stage:

@torch.jit.script
def convert_to_single_emb(x, offset: int = 512):
    feature_num = x.size(1) if len(x.size()) > 1 else 1
    feature_offset = 1 + torch.arange(0, feature_num * offset, offset, dtype=torch.long)
    x = x + feature_offset
    return x

This seems to increase the values of the features, and leads to errors with the embeddings downstream since the lookup table size is much smaller than say, 51200 (if I have a node feature dimension of 100).

[sidnb13]

Found the reasoning well explained here: microsoft/Graphormer#28