Intuition of Uniform attention and Separate attention

[code-rex1]

❓ Questions and Help

• I have been trying to understand the following two concepts as described in the repo:

1. Uniform attention
2. Separate attention

The description is not really clear to me. Would really appreciate it if you can please give an intuition between this two technique.

• Furthermore, Separate attention has two different types:

1. two sep_diff_encoder_type
2. sep_diff_node_type

Its not clear to me what should I use? Any pointers please.🙏

[AlanSwift]

sep_diff_encoder_type: encoder pipeline: RNNencoder --> GNN encoder. The decoder attends on graph encoder and rnn encoder's results separately.
sep_diff_node_type: for example, there is a graph with 10 nodes with two types: A and B. The decoder attends on different node types separately.
See docs.

[code-rex1]

sep_diff_node_type: for example, there is a graph with 10 nodes with two types: A and B. The decoder attends on different node types separately.

@AlanSwift I see I can specify text content of a node with token. How to definite type for a node?

[AlanSwift]

Please refer to the graph construction component.