creating interventional distributions

[KeepMovingXX]

Hello, thank you for your excellent work. I have some confusion regarding creating interventional distributions. The paper mentions creating a memory bank from which to sample for interventions, but I haven't found relevant code in yours. Can you provide some help to me? Thank you.

[Wuyxin]

Hi,

We did not write the memory bank explicitly, rather, we dynamically sample the interventional subgraphs in each batch: https://github.com/Wuyxin/DIR-GNN/blob/main/train/sst2_dir.py#L150

For each subgraph, we create one interventional distribution within the same batch. Another implementation will be sampling a fixed number of subgraphs in each epoch, which will be used for creating the interventional distributions in the next epoch. In that case, the memory bank will be static but less updated to the changes in the current optimization.

Thanks!

[KeepMovingXX]

Thank you for your friendly and patient response.
I have carefully debugged your code. It appears that the intervention is performed on the graph embedding. Given a batch, after reaching the causal_rep and the conf_rep, each conf_rep is iterated over to intervene with the causal_rep using each environment. This means that the causal_rep for a single batch is intervened with for a total of batch_size times. Do I understand this correctly?
Thank you very much！

[Wuyxin]

Yes exactly, this is a very nice summary. Operating interventions on graph structures could be pretty tricky and lead to noise, therefore we conduct intervention in the embedding space instead!

[KeepMovingXX]

Thank you very much!