aravindsankar28/DySAT

DySAT w/o temporal attention is better in Enron

wondergo2017 opened this issue · 0 comments

Thanks for your contribution. I find that DySAT w/o temporal attention is better in Enron.

I change code to delete Temporal Attention part in file models.py.

# 5: Temporal Attention forward
        temporal_inputs = structural_outputs
        outputs= temporal_inputs
        # for temporal_layer in self.temporal_attention_layers:
        #     outputs = temporal_layer(temporal_inputs)  # [N, T, F]
        #     temporal_inputs = outputs
        #     self.attn_wts_all.append(temporal_layer.attn_wts_all)
        return outputs

And run
python train.py --dataset Enron_new --time_steps 16

get results w/o temporal attention :

default results (val) [0.8700378071833649, 0.8700378071833649]
default results (val) [0.8851606805293006, 0.8851606805293006]
default results (test) [0.90290357641944, 0.90290357641944]
default results (test) [0.9008850473577972, 0.9008850473577972]

while DySat with temporal attention has results

default results (val) [0.9040642722117203, 0.9040642722117203]                                                                                                                          
default results (val) [0.9064272211720227, 0.9064272211720227]                                                                                                                          
default results (test) [0.8758863412866829, 0.8758863412866829]                                                                                                                         
default results (test) [0.8781636561254593, 0.8781636561254593]

It seems that DySAT w/o temporal attention performs better than one with temporal attention. Is there something wrong ?