Awesome Attention-based GNNs

A collection of resources on attention-based graph neural networks(Github，Paper).
Welcome to submit a pull request to add more awesome papers.

- [x] [journal] [model] paper_title [[paper]](link) [[code]](link)

Surveys
GRANs : (Graph Recurrent Attention Networks)
GATs : (Graph Attention Networks：Intra-layer， Inter-layer)
Graph Transformers : (Graph Transformers)

Survey

GNN Survey

[Artif Intell Rev2023] [survey] Attention-based Graph Neural Networks: A Survey [paper]
[TKDD2019] [survey] Attention Models in Graphs: A Survey [paper]
[arXiv2022] [survey] Transformer for Graphs: An Overview from Architecture Perspective [paper]
[TIST2022] [] Graph neural networks:Taxonomy, advances, and trends [paper]
[2020] [survey] A Comprehensive Survey on Graph Neural Networks [paper]
[2021] [survey] Graph Deep Learning: State of the Art and Challenges [paper]

Application

[2022] [survey] Graph Neural Networks for Natural Language Processing: A Survey [paper]
[2022] [] Graph neural networks for recommender system [paper]
[2021] [survey] Graph Neural Network for Traffic Forecasting: A Survey [paper]
[2022] [survey] Graph Neural Networks in IoT: A Survey [paper]
[2023] [] Multimodal learning with graphs [paper]

Attention Survey

GRANs

GRU Attention

[ICLR2016] [GGNN] Gated Graph Sequence Neural Networks [paper] [code]
[UAI2018] [GaAN] GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs [paper] [code]
[ICML2018] [GraphRNN] Graphrnn: Generating realistic graphs with deep auto-regressive models [paper] [code]
[NeurIPS2019] [GRAN] Efficient Graph Generation with Graph Recurrent Attention Networks [paper] [code]
[T-SP2020] [GRNN] Gated Graph Recurrent Neural Networks [paper]
[2021] [GR-GAT] Gated Relational Graph Attention Networks [paper]

LSTM Attention

[NeurIPS2017] [GraphSage] Inductive representation learning on large graphs [paper] [code]
[ICML2018] [JK-Net] Representation Learning on Graphs with Jumping Knowledge Networks [paper]
[KDD2018] [GAM] Graph Classification using Structural Attention [paper] [code]
[AAAI2019] [GeniePath] GeniePath: Graph Neural Networks with Adaptive Receptive Paths [paper]

GATs（Intra-layer）

Neighbor Attention

High-Order Attention

[IJCAI2019] [SPAGAN] SPAGAN: Shortest Path Graph Attention Network [paper]
[PKDD2021] [PaGNN] Inductive Link Prediction with Interactive Structure Learning on Attributed Graph [paper]
[arXiv2019] [DeepLinker] Link Prediction via Graph Attention Network [paper]
[IJCNN2020] [CGAT] Heterogeneous Information Network Embedding with Convolutional Graph Attention Networks [paper]
[ICLR2020] [ADSF] Adaptive Structural Fingerprints for Graph Attention Networks [paper]
[KDD2021] [T-GAP] Learning to Walk across Time for Interpretable Temporal Knowledge Graph Completion [paper] [code]
[NeurIPS2018] [MAF] Modeling Attention Flow on Graphs [paper]
[IJCAI2021] [MAGNA] Multi-hop Attention Graph Neural Network [paper] [code]

Relation-Aware Attention

Attention Sampling/Pooling

[NeurIPS2018] [GAW] Watch your step: learning node embeddings via graph attention [paper]
[TPAMI2021] [NLGAT] Non-Local Graph Neural Networks [paper]
[NeurIPS2019] [ChebyGIN] Understanding Attention and Generalization in Graph Neural Networks [paper] [code]
[ICML2019] [SAGPool] Self-Attention Graph Pooling [paper] [code]
[ICCV2019] [Attpool] Attpool: Towards hierarchical feature representation in graph convolutional networks via attention mechanism [paper]

Hierarchical Attention

Hyper-Attention

[ICLR2020] [Hyper-SAGNN] Hyper-Sagnn: A Self-Attention based Graph Neural Network for Hypergraphs [paper] [code]
[CIKM2021] [HHGR] Double-Scale Self-Supervised Hypergraph Learning for Group Recommendation [paper] [code]
[ICDM2021] [HyperTeNet] HyperTeNet: Hypergraph and Transformer-based Neural Network for Personalized List Continuation [paper] [code]
[PR2020] [Hyper-GAT] Hypergraph Convolution and Hypergraph Attention [paper] [code]
[CVPR2020] [] Hypergraph attention networks for multimodal learning [paper]

Inter-Layer GATs

Multi-Level Attention

[KDD2020] [DAGNN] Towards Deeper Graph Neural Networks [paper] [code]
[T-NNLS2020] [AP-GCN] Adaptive propagation graph convolutional network [paper] [code]
[CIKM2021] [TDGNN] Tree Decomposed Graph Neural Network [paper] [code]
[arXiv2020] [GMLP] GMLP: Building Scalable and Flexible Graph Neural Networks with Feature-Message Passing [paper]
[KDD2022] [GAMLP] Graph Attention Multi-layer Perceptron [paper] [code]

Multi-Channel Attention

[AAAI2021] [FAGCN] Beyond Low-frequency Information in Graph Convolutional Networks [paper] [code]
[arXiv2021] [ACM] Is Heterophily A Real Nightmare For Graph Neural Networks To Do Node Classification? [paper]

Multi-View Attention

[KDD2020] [AM-GCN] AM-GCN: Adaptive Multi-channel Graph Convolutional Networks [paper] [code]
[AAAI2021] [UAG] Uncertainty-aware Attention Graph Neural Network for Defending Adversarial Attacks [paper]
[CIKM2021] [MV-GNN] Semi-Supervised and Self-Supervised Classification with Multi-View Graph Neural Networks [paper]
[KSEM2021] [GENet] Graph Ensemble Networks for Semi-supervised Embedding Learning [paper]
[NN2020] [MGAT] MGAT: Multi-view Graph Attention Networks [paper]
[CIKM2017] [MVE] An Attention-based Collaboration Framework for Multi-View Network Representation Learning [paper]
[NC2021] [EAGCN] Multi-view spectral graph convolution with consistent edge attention for molecular modeling [paper]
[SIGIR2020] [GCE-GNN] Global Context Enhanced Graph Neural Networks for Session-based Recommendation [paper]

Spatio-Temporal Attention

Time Series Attention

[ICLR2022] [RainDrop] Graph-Guided Network for Irregularly Sampled Multivariate Time Series [paper]
[ICLR2021] [mTAND] Multi-Time Attention Networks for Irregularly Sampled Time Series [paper] [code]
[ICDM2020] [MTAD-GAT] Multivariate Time-series Anomaly Detection via Graph Attention Network [paper]
[WWW2020] [GACNN] Towards Fine-grained Flow Forecasting: A Graph Attention Approach for Bike Sharing Systems [paper]
[IJCAI2018] [GeoMAN] GeoMAN: Multi-level Attention Networks for Geo-sensory Time Series Prediction [paper]

GraphTransformers

[journal] [model] paper_title [paper] [code]

Citation

Sun, C., Li, C., Lin, X. et al. Attention-based graph neural networks: a survey. Artif Intell Rev (2023). https://doi.org/10.1007/s10462-023-10577-2

Bib

@article{ WOS:001051847200001,
   Author = {Sun, Chengcheng and Li, Chenhao and Lin, Xiang and Zheng, Tianji and Meng, Fanrong and Rui, Xiaobin and Wang, Zhixiao},
   Title = {Attention-based graph neural networks: a survey},
   Journal = {ARTIFICIAL INTELLIGENCE REVIEW},
   Year = {2023},
   Month = {2023 AUG 21},
   DOI = {10.1007/s10462-023-10577-2},
   EarlyAccessDate = {AUG 2023},
   ISSN = {0269-2821},
   EISSN = {1573-7462},
   Unique-ID = {WOS:001051847200001},
}

sunxiaobei/awesome-attention-based-gnns

Awesome Attention-based GNNs

Table of Contents

Survey

GNN Survey

Application

Attention Survey

GRANs

GRU Attention

LSTM Attention

GATs（Intra-layer）

Neighbor Attention

High-Order Attention

Relation-Aware Attention

Attention Sampling/Pooling

Hierarchical Attention

Hyper-Attention

Inter-Layer GATs

Multi-Level Attention

Multi-Channel Attention

Multi-View Attention

Spatio-Temporal Attention

Time Series Attention

GraphTransformers

Next

Citation

Bib