scigeek72/GNN_Repo

Contains reference papers and other info related to Graph Neural Network

GNN_Repo

Contains reference papers and other info related to Graph Neural Network

Papers (in random order)

• Benchmarking Node Outlier Detection on Graphs by Liu et.al

• Graph Representation Learing in Biomedicine by Li, Huang, Zitnik

• Modeling Relational Data with Graph Convolutional Network By Schlichtkrull, Kipf, Welling, Bloom, Titov, van der Berg

• Fast Graph Representation Learning with PyTorch-Geometric by Mathew Fay et.al

• Graph Convolutional Networks for Graph with Multi-dimensionally weighted edges by Chen (July 20, 2020)

• Strategies for Pre-Training Graph Neural Networks by Hu, Liu, Gomes, Zitnik, Liang, Pande, and Leskovec

  • Interesting paper on pretraining strategies for performance enhancement
  • Has implementation in PyG (look here)

• ETA prediction using Graph Neural Networks in Google Maps

  • Very dense paper with a lot of innovations, includes edge-features.

• Graph Convolutional Neural Nets for Web-Scale Recommender System (PinSage) by Ying, He, Chen, Eksombatchai, Hamilton, Leskovec (2018)

  • PinSage is implemented and productionized at Pinterest. Graph Size is 3 billion Nodes and 18 billion Edges
  • This paper has number of improvements on traditional GCN and GraphSage

• Identity Aware Graph Neural Nets (ID-GNN) by You, Gomes-Selman, Ying, Leskovec (2021)

  • More general framework than GNN (see Lecture 16 in CS224W course) and perform better for node/edge/graph level tasks.
  • Can be implemented with PyG/DGL etc. But more complicated. So, first start out with the original GNN.

• Design Space for Graph Neural Networks by You, Ying, Leskovec (2021)

• Learning Structural Node Embeddings via Diffusion Wavelets by Donnat, Zitnik, Hallac, Leskovec(2018)

  • Node embeddings that also incorporate structural properties of the nodes (such as hubs). Very Interesting but may require more work.

• Exploiting Edge features in Graph Neural Networks

• Neural Message Passing in Quantum Chemistry by Gilmer et.al

  • Consider Message Passing using edge-features

• Relational inductive biases, deep learning, and graph networks by . Battaglia et.al

  • Consider Message passing using edge-features

• NENN: Incorporate Edge and Node Features in Graph Neural Networks by Yang, Li (2020)

  • This is attention based algorithm. But they seems to be doing the right updates as opposed to E-GraphSAGE. Can we incorporate this idea in the way GraphSAGE does it but instead now we incorporates (and updates) edge-features as well?????

• Principal Neighborhood Aggregation for graph nets

• DeepWalk: Online Learning of Social Representation

• Node2vec: Scalable Feature Learning For Networks

• GraphSAGE: Inductive Representation Learning on Large Graphs by Hamilton, Ying, Leskovec

  • Reviews are good. Inductive and not transductive can work on previously unseen nodes.
  • No edge-features were considered for Node embedding (so, look at the following paper in this list).
  • Use this algorithm. It can be trained in uspervised mode.

• E-GraphSAGE: A Graph Neural Networks based Intrusion Detection System for IoT

  • On computer networks
  • There seems to be a fundamental problem in the updates of nodes features using edge features (line 5 in the algorithm). It talks about edge-features from the previous layer (k-1), but never updates edge-features.
  • DON'T USE IT AS IS. MODIFY IT.

• Graph Attention Networks

• Semi-Supervised Classification with Graph Convolutional Networks by Kipf, Welling (2017)

• Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering by Defferrard, Bresson, Vandergheynst (2017)

• NF-GNN:Network Flow Graph for Malware Detection and classification

  • On computer networks
  • Incorporates Edge-features in the model

• Residual Gated Graph ConvNets

• Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties

  • Incorporates EDGE-features in the model
  • Github repo (with Pytorch)

• Edge Attention based Multi-relational Graph Convolutional Networks

  • An extension of the Graph Attention Network (see blog: Graph attention network for more comments)

• How powerful are Graph Neural Networks? by Leskovec, Jagelka

  • assess different aggregation functions (excellent review on Openreview.net)

• UBER-GNN: A User-Based Embeddings Recommendation based on Graph Neural Networks (Uber Eats Recommender System) by Huang, Bi, Wu, Wang, Xiao (2020)

• Variational Graph Auto-encoders by Kipf, Welling

Survey Papers

• A comprehensive Survey on Graph Neural Networks
• Graph Neural Networks: A review of methods and application

Books and Book Chapters

• Graph Reprenstation Learning by William Hamilton
• Graph Neural Networks: Foundation, Frontiers and Applications
  • All the chapters are available (including on Anomaly detection which I am listing below seperately) written some of the leading researchers in the field
• Book Chapter Graph Neural Networks in Anomaly Detection by Shen Wang, Philip S. Yu
• Geometric Deep Learning by Bronstein, Bruna, Cohen, and Veličković4
  • Note: Very theoratical and mathy.

Courses, Tutorials and Blog posts

• CS224W:Machine Learning with Graphs
• Pytorch Geometric Module
  • Contains a good assemblage of links to tutorials and blogs. Particularly look at these two Colab Notebooks and Video Tutorials and External Resources
• Blog: A gentle introduction to Graph Neural Networks
  • Nice introductory explanation with pointers to actual papers
• Blog: Understanding Convolution on Graphs by Ameya Daigavane, Balaraman Ravindran, Gaurav Aggarwal
  • Nice explanation (how Polynomial filters (Graph Laplacian) are translated into Message passing concepts*. Also have important pointers to papers
• Blog: Understanding Graph Neural Networks by Irhum Safkat
  • Nice introductory explanation
• Blog: Graph Attention Networks by Petar Veličković
• Blog: Graph Convolutional Networks by Thomas Kipf
• Blog: Food Discovery with Uber Eats: Using Graph Learning to Power Recommendations by Jain, Liu, Sarda, Molino (2019)
  • For the actual paper, see the paper section above
• Blog: Best GNN architectures by Sergios Karagiannakos
  • Note: Its a nice little introduction. Moreover, it has a good set of resources for other ML stuff like Computer Vision. Check it out.

Related Videos

• Youtube: How to use edge-features in GNN
  • Nice visualization and summary of the important message passing concepts that incorporate multi-dimensional Edge features from various papers (including pytorch modules and how to incorporate edge-features/edge-attributes into GNN code)
• Youtube: Geometric Deep Learning on Graphs by Michael Bronstein
• Youtube: Intro to GNN by Petar Veličković
• Youtube: Intro to GNN: Model and Applications by Microsoft Research (Not very informative)

Python Libraries to work with Graphs

• PyGOD
  • PyGOD is based on Pytorch and pytorch_geometric and used for graph outlier detection(anomaly detection)
• DGL Documentation and DGL code base
  • Can be used with Pytorch, Tensorflow and MaxNet. Try this First
• GraphNets
  • From DeepMind. Tensorflow based high quality modules (try to use this instead of Stellar Graphs).
• Stellar Graphs
  • Tensorflow + Keras based.
• Pytorch Geometric
  • Pytorch based

How to form a graph in Python?

• Blog Working With Large Internal Link Graphs in Python by Justin
• Networkx Documentation

Other Resources

• DeepSnap
• GraphGym and PyG
  • Based on Pytorche. PyG is a more tightly integrated version of GraphGym
• SNAP: Stanford Network Analysis Project

CyberSecurity Datasets

• CyberSecurity Datasets
  • Look at CICandMal-2017 dataset (and other similar datasets)
• Graph Datasets