This open-source library is available to summarize several years of research papers on graph reinforcement learning for the convenience of researchers.
For any ideas and literature on graph reinforcement learning, please contact me.
Mingshuo Nie,
Northeastern University, China.
E: niemingshuo@stumail.neu.edu.cn
If you find this work useful in your research, please consider citing:
@article{mingshuo2022reinforcement, title={Reinforcement Learning on Graph: A Survey}, author={Mingshuo, Nie and Dongming, Chen and Dongqi, Wang}, journal={arXiv preprint arXiv:2204.06127}, year={2022} }
All the reinforcement learning methods used in the literature are as follows.
| RL method | Abbr. | Year | Paper |
|---|---|---|---|
| Markov Decision Process | MDP | \ | \ |
| Monte Carlo Tree Search | MCTS | \ | \ |
| Bernoulli Multi-armed Bandit | BMAB | 2005 | Paper |
| Q-learning | QL | 1992 | Paper |
| Deep Q-learning Network | DQN | 2015 | Paper |
| Double DQN | DDQN | 2016 | Paper |
| Cascaded DQN | CDQN | 2019 | Paper |
| Actor-Critic | AC | 1999 | Paper |
| Advantage Actor-Critic | A2C | 2016 | Paper |
| Asynchronous Advantage Actor-Critic | A3C | 2016 | Paper |
| Deep Deterministic Policy Gradient | DDPG | 2016 | Paper |
| proximal policy optimization | PPO | 2017 | Paper |
| neural fitted Q-iteration | NFQI | 2005 | Paper |
| REINFORCE | REINFORCE | 1992 | Paper |
| Year | Venue | Model | Title | Algorithm | Paper | Code |
|---|---|---|---|---|---|---|
| 2022 | IEEE TPAMI | DRL-DBSCAN | Reinforced, Incremental and Cross-lingual Event Detection From Social Messages | MarGNN | Paper | Code |
| 2022 | IEEE TKDE | RTGNN | Multi-view Tensor Graph Neural Networks Through Reinforced Aggregation | MDP | Paper | Code |
| 2022 | IEEE TKDE | LUCE | Lifelong Property Price Prediction: A Case Study for the Toronto Real Estate Market | MDP | Paper | Code |
| 2022 | arXiv | BN-GNN | Deep Reinforcement Learning Guided Graph Neural Networks for Brain Network Analysis | DDQN | Paper | \ |
| 2022 | ICLR | G2RL | Graph-Enhanced Exploration for Goal-oriented Reinforcement Learning | QL | Paper | \ |
| 2022 | ICLR | AGILE | Know Your Action Set: Learning Action Relations for Reinforcement Learning | PPO\DQN\CDQN | Paper | Code |
| 2022 | ICLR | MAPSRL-2 | Reinforcement Learning under a Multi-agent Predictive State Representation Model: Method and Theory | QL | Paper | \ |
| 2022 | ICLR | SWAT | Structure-Aware Transformer Policy for Inhomogeneous Multi-Task Reinforcement Learning | AC | Paper | \ |
| 2022 | Knowledge-Based Systems | RF | Dynamic knowledge graph reasoning based on deep reinforcement learning | AC | Paper | \ |
| 2022 | arXiv | two-step hybrid RL | Learning Two-Step Hybrid Policy for Graph-Based Interpretable Reinforcement Learning | MDP | Paper | \ |
| 2022 | arXiv | GTA-RL | Solving Dynamic Graph Problems with Multi-Attention Deep Reinforcement Learning | REINFORCE | Paper | Code |
| 2022 | arXiv | AdRumor-RL | Interpretable and Effective Reinforcement Learning for Attacking against Graph-based Rumor Detection | DQN | Paper | \ |
| 2022 | arXiv | GraphAug | Automated Data Augmentations for Graph Classification | MDP | Paper | \ |
| 2022 | Applied Intelligence | RLPath | RLPath: a knowledge graph link prediction method using reinforcement learning based attentive relation path searching and representation learning | MDP | Paper |
| Year | Venue | Model | Title | Algorithm | Paper | Code |
|---|---|---|---|---|---|---|
| 2021 | IEEE ICDM | ACE-HGNN | ACE-HGNN: Adaptive Curvature Exploration Hyperbolic Graph Neural Network | Nash Q-leaning | Paper | \ |
| 2021 | ICML | SubgraphX | On Explainability of Graph Neural Networks via Subgraph Explorations | MCTS | Paper | Code |
| 2021 | WWW | SUGAR | SUGAR: Subgraph Neural Network with Reinforcement Pooling and Self-Supervised Mutual Information Mechanism | QL | Paper | Code |
| 2021 | IJCAI | CORL | Ordering-Based Causal Discovery with Reinforcement Learning | MDP | Paper | Code |
| 2021 | ACM Transactions on Information Systems | RioGNN | Reinforced Neighborhood Selection Guided Multi-Relational Graph Neural Networks | MDP | Paper | Code |
| 2021 | ICML | RLGN | Controlling Graph Dynamics with Reinforcement Learning and Graph Neural Networks | PPO | Paper | \ |
| 2021 | arXiv | RL | Reinforcement Learning for Flexibility Design Problems | MDP | Paper | \ |
| 2021 | Computer‐Aided Civil and Infrastructure Engineering | GCQ | Graph neural network and reinforcement learning for multi-agent cooperative control of connected autonomous vehicles | DQN | Paper | \ |
| 2021 | International Journal of Production Research | Park et al. | Learning to schedule job-shop problems: representation and policy learning using graph neural network and reinforcement learning | PPO | Paper | \ |
| 2021 | Information Sciences | Dynamic graph | Dynamic graph convolutional network for long-term traffic flow prediction with reinforcement learning | PPO | Paper | \ |
| 2021 | 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) | AttnPath | Incorporating Graph Attention Mechanism into Knowledge Graph Reasoning Based on Deep Reinforcement Learning | MDP | Paper | \ |
| 2021 | IJCAI | RLH | Reasoning like human: Hierarchical reinforcement learning for knowledge graph reasoning | MDP | Paper | \ |
| 2021 | IEEE Communications Letters | DeepOpt | Combining Deep Reinforcement Learning With Graph Neural Networks for Optimal VNF Placement | REINFORCE | Paper | \ |
| 2021 | ACM SIGIR | UNICORN | Unified conversational recommendation policy learning via graph-based reinforcement learning | DDQN | Paper | \ |
| 2021 | IEEE Transactions on Intelligent Transportation Systems | IG-RL | IG-RL: Inductive Graph Reinforcement Learning for Massive-Scale Traffic Signal Control | MDP | Paper | Code |
| 2021 | Neurocomputing | MGRL | MGRL: Graph neural network based inference in a Markov network with reinforcement learning for visual navigation | A2C | Paper | \ |
| 2021 | IEEE Transactions on Intelligent Transportation Systems | SAGE-Garph | Deep Reinforcement Learning With Graph Representation for Vehicle Repositioning | DDQN | Paper | \ |
| 2021 | arXiv | TITer | TimeTraveler: Reinforcement Learning for Temporal Knowledge Graph Forecasting | REINFORCE | Paper | Code |
| 2021 | IEEE Internet of Things Journal | GRLO | Graph Reinforcement Learning based Task offloading for Multi-access Edge Computing | AC | Paper | \ |
| 2021 | arXiv | SparRL | SparRL: Graph Sparsification via Deep Reinforcement Learning | MDP | Paper | Code |
| 2021 | The 10th International Joint Conference on Knowledge Graphs | PAAR | Multi-hop Knowledge Graph Reasoning Based on Hyperbolic Knowledge Graph Embedding and Reinforcement Learning | MDP | Paper | Code |
| 2021 | arXiv | Vulcan | Vulcan: Solving the Steiner Tree Problem with Graph Neural Networks and Deep Reinforcement Learning | DDQN | Paper | \ |
| 2021 | KSEM | Zheng et al. | Hierarchical Policy Network with Multi-agent for Knowledge Graph Reasoning Based on Reinforcement Learning | REINFORCE | Paper | \ |
| Year | Venue | Model | Title | Algorithm | Paper | Code |
|---|---|---|---|---|---|---|
| 2020 | KDD | Policy-GNN | Policy-GNN: Aggregation Optimization for Graph Neural Networks | DQN | Paper | Code |
| 2020 | IJCAI | eGCN | Dynamic Electronic Toll Collection via Multi-Agent Deep Reinforcement Learning with Edge-Based Graph Convolutional Networks | MDP | Paper | \ |
| 2020 | WWW | NIPA | Adversarial Attacks on Graph Neural Networks via Node Injections: A Hierarchical Reinforcement Learning Approach | DQN | Paper | \ |
| 2020 | CIKM | CARE-GNN | Enhancing Graph Neural Network-based Fraud Detectors against Camouflaged Fraudsters | BMAB | Paper | Code |
| 2020 | arXiv | RL-HGNN | Reinforcement Learning Enhanced Heterogeneous Graph Neural Network | DQN | Paper | \ |
| 2020 | ICLR | RL-BIC | Causal Discovery with Reinforcement Learning | AC | Paper | Code |
| 2020 | ICLR | RL-based Graph2Seq | Reinforcement Learning Based Graph-to-Sequence Model for Natural Question Generation | REINFORCE | Paper | Code |
| 2020 | IEEE Journal on Selected Areas in Communications | A3C+GCN | Automatic Virtual Network Embedding: A Deep Reinforcement Learning Approach With Graph Convolutional Networks | A3C | Paper | \ |
| 2020 | ICLR | DGN | Graph Convolutional Reinforcement Learning | QL | Paper | Code |
| 2020 | ACM SIGIR | KGQR | Interactive Recommender System via Knowledge Graph-enhanced Reinforcement Learning | DQN | Paper | \ |
| 2020 | Journal of cheminformatics | DeepGraphMolGen | DeepGraphMolGen, a multi-objective, computational strategy for generating molecules with desirable properties: a graph convolution and reinforcement learning approach | PPO | Paper | Code |
| 2020 | 57th ACM/IEEE Design Automation Conference (DAC) | GCN-RL | GCN-RL Circuit Designer: Transferable Transistor Sizing with Graph Neural Networks and Reinforcement Learning | AC | Paper | \ |
| 2020 | arXiv | KG-A2C | Graph Constrained Reinforcement Learning for Natural Language Action Spaces | A2C | Paper | Code |
| 2020 | ACM SIGKDD | IMUP | Incremental Mobile User Profiling: Reinforcement Learning with Spatial Knowledge Graph for Modeling Event Streams | DQN | Paper | \ |
| 2020 | Knowledge-Based Systems | ADRL | ADRL: An attention-based deep reinforcement learning framework for knowledge graph reasoning | AC | Paper | \ |
| 2020 | Knowledge-Based Systems | GRL | GRL: Knowledge graph completion with GAN-based reinforcement learning | DDPG | Paper | \ |
| 2020 | IEEE Access | NAKASHIMA et al. | Deep Reinforcement Learning-Based Channel Allocation for Wireless LANs With Graph Convolutional Networks | DDQN | Paper | \ |
| 2020 | IEEE Access | SILVA et al. | Temporal Graph Traversals Using Reinforcement Learning With Proximal Policy Optimization | PPO | Paper | \ |
| 2020 | IEEE Access | Wang et al. | Risk-Aware Identification of Highly Suspected COVID-19 Cases in Social IoT: A Joint Graph Theory and Reinforcement Learning Approach | Q-learning | Paper | \ |
| 2020 | KDD | XGNN | XGNN: Towards Model-Level Explanations of Graph Neural Networks | MDP | Paper | \ |
| 2020 | NeurIPS | GPA | Graph Policy Network for Transferable Active Learning on Graphs | MDP | Paper | Code |
| 2020 | AAAI/ACM Conference on AI | GAEA | GAEA: Graph Augmentation for Equitable Access via Reinforcement Learning | MDP | Paper | Code |
| Year | Venue | Model | Title | Algorithm | Paper | Code |
|---|---|---|---|---|---|---|
| 2019 | CIKM | CompNet | Order-free Medicine Combination Prediction with Graph Convolutional Reinforcement Learning | DQN | Paper | Code |
| 2019 | AISTATS | GRPI | Representation Learning on Graphs: A Reinforcement Learning Application | MDP | Paper | Code |
| 2019 | arXiv | DRL+GNN | Deep Reinforcement Learning meets Graph Neural Networks: exploring a routing optimization use case | DQN | Paper | Code |
| 2019 | arXiv | AGNN | Auto-GNN: Neural Architecture Search of Graph Neural Networks | REINFORCE | Paper | \ |
| 2019 | NeurIPS | GMETAEXP | Learning Transferable Graph Exploration | MDP | Paper | \ |
| 2019 | KDD | GTPN | Graph Transformation Policy Network for Chemical Reaction Prediction | A2C | Paper | \ |
| 2019 | arXiv | GPN | Combinatorial Optimization by Graph Pointer Networks and Hierarchical Reinforcement Learning | REINFORCE | Paper | Code |
| 2019 | IEEE Transactions on Network and Service Management | DDPG-HFA | A Deep Reinforcement Learning Approach for VNF Forwarding Graph Embedding | DDPG | Paper | \ |
| 2019 | ICDE | RQL | Adaptive Dynamic Bipartite Graph Matching: A Reinforcement Learning Approach | QL | Paper | \ |
| 2019 | Acta Astronautica | Das-Stuart et al. | Rapid trajectory design in complex environments enabled by reinforcement learning and graph search strategies | MDP | Paper | \ |
| 2019 | arXiv | Ekar | Ekar: An Explainable Method for Knowledge Aware Recommendation | MDP | Paper | \ |
| 2019 | arXiv | RL-VAE | Decoding Molecular Graph Embeddings with Reinforcement Learning | MDP | Paper | \ |
| 2019 | International Symposium on Problems of Redundancy in Information and Control Systems (RED) | MAGNet | MAGNet: Multi-agent Graph Network for Deep Multi-agent Reinforcement Learning | AC | Paper | \ |
| 2019 | ACM SIGIR | PGPR | Reinforcement Knowledge Graph Reasoning for Explainable Recommendation | REINFORCE | Paper | Code |
| 2019 | arXiv | NIPA | Node Injection Attacks on Graphs via Reinforcement Learning | DQN | Paper | \ |
| 2019 | DRL4KDD | Rel4KC | Rel4KC: A Reinforcement Learning Agent for Knowledge Graph Completion and Validation | MDP | Paper | Code |
| 2019 | arXiv | ReWatt | Attacking Graph Convolutional Networks via Rewiring | MDP | Paper | \ |
| 2019 | ICDM | GDPNet | Learning Robust Representations with Graph Denoising Policy Network | MDP | Paper | \ |
| Year | Venue | Model | Title | Algorithm | Paper | Code |
|---|---|---|---|---|---|---|
| 2018 | arXiv | DGN | Graph Convolutional Reinforcement Learning for Multi-Agent Cooperation | DQN | Paper | Code |
| 2018 | ICML | RL-S2V | Adversarial Attack on Graph Structured Data | Q-learning | Paper | - |
| 2018 | NeurIPS | GCPN | Graph convolutional policy network for goal-directed molecular graph generation | MDP | Paper | Code |
| 2018 | ICLR | NerveNet | NerveNet: Learning Structured Policy with Graph Neural Networks | PPO | Paper | \ |
| 2018 | arXiv | KG-DQN | Playing Text-Adventure Games with Graph-Based Deep Reinforcement Learning | DQN | Paper | Code |
| 2018 | COLING | Chen et al. | Structured Dialogue Policy with Graph Neural Networks | REINFORCE | Paper | \ |
| 2018 | arXiv | Hamrick et al. | Relational inductive bias for physical construction in humans and machines | Q-learning | Paper | \ |
| 2018 | AAAI | ASNets | Action Schema Networks: Generalised Policies with Deep Learning | MDP | Paper | Code |
| 2018 | PMLR | Zhang et al. | Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents. | Actor-Critic | Paper | \ |
| 2018 | International Conference on Intelligent Transportation | NFQI | Traffic Signal Control Based on Reinforcement Learning with Graph Convolutional Neural Nets | NFQI | Paper | \ |
| 2018 | IEEE International Conference on Data Mining Workshops (ICDMW) | MARLPaR | Path Reasoning over Knowledge Graph: A Multi-agent and Reinforcement Learning Based Method | MDP | Paper | \ |
| 2018 | IEEE International Conference on Big Data (Big Data) | Obara et al. | Deep Reinforcement Learning Approach for Train Rescheduling Utilizing Graph Theory | DQN | Paper | \ |
| 2018 | ICLR | ReinforceWalk | ReinforceWalk: Learning to Walk in Graph with Monte Carlo Tree Search | MCTS | Paper | \ |
| 2018 | Conference on Empirical Methods in Natural Language Processing | Lin et al. | Multi-Hop Knowledge Graph Reasoning with Reward Shaping | REINFORCE | Paper | \ |
| 2018 | arXiv | MolGAN | MolGAN: An implicit generative model for small molecular graphs | MDP | Paper | \ |
| 2018 | ACM SIGKDD | GAM | Graph Classification using structural attention | Partially Observable Markov Decision Process (POMDP) | Paper | \ |
| List | Year | Venue | Model | Title | Algorithm | Paper | Code |
|---|---|---|---|---|---|---|---|
| o | 2017 | NIPS | S2V-DQN | Learning Combinatorial Optimization Algorithms over Graphs | QL | Paper | Code |
| o | 2017 | arXiv | Deeppath | Deeppath: A reinforcement learning method for knowledge graph reasoning | DQN | Paper | Code |
| o | 2017 | ICLR | MINERVA | Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning | REINFORCE | Paper | Code |
| o | 2017 | arXiv | KBGAN | KBGAN: Adversarial Learning for Knowledge Graph Embeddings | REINFORCE | Paper | Code |