A curated list of papers and resources about large language models (LLMs) on graphs based on our survey paper: Large Language Models on Graphs: A Comprehensive Survey.
This repo will be continuously updated. Don't forget to star it and keep tuned!
Please cite the paper in Citations if you find the resource helpful for your research. Thanks!
Large language models (LLMs), such as ChatGPT and LLaMA, are creating significant advancements in natural language processing, due to their strong text encoding/decoding ability and newly found emergent capability (e.g., reasoning). While LLMs are mainly designed to process pure texts, there are many real-world scenarios where text data are associated with rich structure information in the form of graphs (e.g., academic networks, and e-commerce networks) or scenarios where graph data are captioned with rich textual information (e.g., molecules with descriptions). Besides, although LLMs have shown their pure text-based reasoning ability, it is underexplored whether such ability can be generalized to graph scenarios (i.e., graph-based reasoning). In this paper, we provide a comprehensive review of scenarios and techniques related to large language models on graphs.
- Awesome-Language-Model-on-Graphs
The Transformer architecture used in the work, e.g., EncoderOnly, DecoderOnly, EncoderDecoder.
The size of the large language model, e.g., medium (i.e., less than 1B parameters), LLM (i.e., more than 1B parameters).
-
Unifying Large Language Models and Knowledge Graphs: A Roadmap.
preprint
Shirui Pan, Linhao Luo, Yufei Wang, Chen Chen, Jiapu Wang, Xindong Wu [PDF], 2023.6
-
Integrating Graphs with Large Language Models: Methods and Prospects
preprint
Shirui Pan, Yizhen Zheng, Yixin Liu [PDF], 2023.10
-
Towards graph foundation models: A survey and beyond.
preprint
Jiawei Liu, Cheng Yang, Zhiyuan Lu, Junze Chen, Yibo Li, Mengmei Zhang, Ting Bai, Yuan Fang, Lichao Sun, Philip S. Yu, Chuan Shi. [PDF], 2023.10
-
A Survey of Graph Meets Large Language Model: Progress and Future Directions.
preprint
Yuhan Li, Zhixun Li, Peisong Wang, Jia Li, Xiangguo Sun, Hong Cheng, Jeffrey Xu Yu. [PDF], 2023.11
Table 3 in our survey paper Large Language Models on Graphs: A Comprehensive Survey.
-
Can Language Models Solve Graph Problems in Natural Language?
preprint
Heng Wang, Shangbin Feng, Tianxing He, Zhaoxuan Tan, Xiaochuang Han, Yulia Tsvetkov. [PDF] [Code], 2023.5,
-
GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarking.
preprint
Jiayan Guo, Lun Du, Hengyu Liu, Mengyu Zhou, Xinyi He, Shi Han. [PDF], 2023.5,
-
Evaluating Large Language Models on Graphs: Performance Insights and Comparative Analysis.
preprint
-
Talk Like A Graph: Encoding Graphs For Large Language Models.
preprint
Bahare Fatemi, Jonathan Halcrow, Bryan Perozzi. [PDF], 2023.10,
-
GraphLLM: Boosting Graph Reasoning Ability of Large Language Model.
preprint
Ziwei Chai, Tianjie Zhang, Liang Wu, Kaiqiao Han, Xiaohai Hu, Xuanwen Huang, Yang Yang. [PDF] [Code], 2023.10,
-
LLM4DyG: Can Large Language Models Solve Problems on Dynamic Graphs?.
preprint
Zeyang Zhang, Xin Wang, Ziwei Zhang, Haoyang Li, Yijian Qin, Simin Wu, Wenwu Zhu [PDF] [Code], 2023.10,
-
Which Modality should I use - Text, Motif, or Image? : Understanding Graphs with Large Language Models.
preprint
Debarati Das, Ishaan Gupta, Jaideep Srivastava, Dongyeop Kang [PDF] [Code], 2023.11,
-
StructGPT: A General Framework for Large Language Model to Reason over Structured Data.
preprint
Jinhao Jiang, Kun Zhou, Zican Dong, Keming Ye, Wayne Xin Zhao, Ji-Rong Wen. [PDF] [Code], 2023.5,
-
Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph.
preprint
Jiashuo Sun, Chengjin Xu, Lumingyuan Tang, Saizhuo Wang, Chen Lin, Yeyun Gong, Lionel M. Ni, Heung-Yeung Shum, Jian Guo. [PDF] [Code], 2023.7,
-
Exploring Large Language Model for Graph Data Understanding in Online Job Recommendations.
preprint
Likang Wu, Zhaopeng Qiu, Zhi Zheng, Hengshu Zhu, Enhong Chen. [PDF] [Code], 2023.7,
-
Knowledge Graph Prompting for Multi-Document Question Answering.
AAAI2024
Yu Wang, Nedim Lipka, Ryan Rossi, Alex Siu, Ruiyi Zhang, Tyler Derr. [PDF] [Code], 2023.8,
-
ChatRule: Mining Logical Rules with Large Language Models for Knowledge Graph Reasoning.
preprint
Linhao Luo, Jiaxin Ju, Bo Xiong, Yuan-Fang Li, Gholamreza Haffari, Shirui Pan. [PDF] [Code], 2023.9,
-
Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning.
preprint
Linhao Luo, Yuan-Fang Li, Gholamreza Haffari, Shirui Pan. [PDF] [Code], 2023.10,
-
Thought Propagation: An Analogical Approach to Complex Reasoning with Large Language Models.
preprint
Junchi Yu, Ran He, Rex Ying. [PDF], 2023.10,
-
Large Language Models Can Learn Temporal Reasoning.
preprint
Siheng Xiong, Ali Payani, Ramana Kompella, Faramarz Fekri. [PDF], 2024.1,
-
Exploring the Limitations of Graph Reasoning in Large Language Models.
preprint
Palaash Agrawal, Shavak Vasania, Cheston Tan. [PDF], 2024.2,
-
Rendering Graphs for Graph Reasoning in Multimodal Large Language Models.
preprint
Yanbin Wei, Shuai Fu, Weisen Jiang, James T. Kwok, Yu Zhang. [PDF], 2024.2,
-
Graph-enhanced Large Language Models in Asynchronous Plan Reasoning.
preprint
Fangru Lin, Emanuele La Malfa, Valentin Hofmann, Elle Michelle Yang, Anthony Cohn, Janet B. Pierrehumbert. [PDF], 2024.2,
-
Microstructures and Accuracy of Graph Recall by Large Language Models.
preprint
Yanbang Wang, Hejie Cui, Jon Kleinberg. [PDF], 2024.2,
-
Structure Guided Prompt: Instructing Large Language Model in Multi-Step Reasoning by Exploring Graph Structure of the Text.
preprint
Kewei Cheng, Nesreen K. Ahmed, Theodore Willke, Yizhou Sun. [PDF], 2024.2,
-
GraphInstruct: Empowering Large Language Models with Graph Understanding and Reasoning Capability.
preprint
Zihan Luo, Xiran Song, Hong Huang, Jianxun Lian, Chenhao Zhang, Jinqi Jiang, Xing Xie, Hai Jin. [PDF], 2024.3,
-
Call Me When Necessary: LLMs can Efficiently and Faithfully Reason over Structured Environments.
preprint
Sitao Cheng, Ziyuan Zhuang, Yong Xu, Fangkai Yang, Chaoyun Zhang, Xiaoting Qin, Xiang Huang, Ling Chen, Qingwei Lin, Dongmei Zhang, Saravan Rajmohan, Qi Zhang. [PDF], 2024.3,
-
Exploring the Potential of Large Language Models in Graph Generation.
preprint
Yang Yao, Xin Wang, Zeyang Zhang, Yijian Qin, Ziwei Zhang, Xu Chu, Yuekui Yang, Wenwu Zhu, Hong Mei. [PDF], 2024.3,
-
Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPT.
preprint
-
Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs.
preprint
Bowen Jin, Chulin Xie, Jiawei Zhang, Kashob Kumar Roy, Yu Zhang, Suhang Wang, Yu Meng, Jiawei Han. [PDF] [Code] [Data], 2024.4,
-
From Local to Global: A Graph RAG Approach to Query-Focused Summarization.
preprint
Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Jonathan Larson. [PDF], 2024.4,
Table 7 in our survey paper Large Language Models on Graphs: A Comprehensive Survey.
-
MATCH: Metadata-Aware Text Classification in A Large Hierarchy.
WWW 2021
Yu Zhang, Zhihong Shen, Yuxiao Dong, Kuansan Wang, Jiawei Han. [PDF] [Code], 2021.2,
-
QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering.
NAACL 2021
Michihiro Yasunaga, Hongyu Ren, Antoine Bosselut, Percy Liang, Jure Leskovec. [PDF] [Code], 2021.4,
-
Natural Language is All a Graph Needs.
preprint
Ruosong Ye, Caiqi Zhang, Runhui Wang, Shuyuan Xu, Yongfeng Zhang. [PDF], 2023.8,
-
Can LLMs Effectively Leverage Graph Structural Information: When and Why.
preprint
Jin Huang, Xingjian Zhang, Qiaozhu Mei, Jiaqi Ma. [PDF] [Code], 2023.9,
-
Graph Neural Prompting with Large Language Models.
preprint
Yijun Tian, Huan Song, Zichen Wang, Haozhu Wang, Ziqing Hu, Fang Wang, Nitesh V. Chawla, Panpan Xu. [PDF], 2023.9,
-
Prompt-based Node Feature Extractor for Few-shot Learning on Text-Attributed Graphs.
preprint
Xuanwen Huang, Kaiqiao Han, Dezheng Bao, Quanjin Tao, Zhisheng Zhang, Yang Yang, Qi Zhu. [PDF], 2023.9, ,
-
GraphText: Graph Reasoning in Text Space.
preprint
Jianan Zhao, Le Zhuo, Yikang Shen, Meng Qu, Kai Liu, Michael Bronstein, Zhaocheng Zhu, Jian Tang [PDF], 2023.10,
-
GraphGPT: Graph Instruction Tuning for Large Language Models.
preprint
Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, Chao Huang. [PDF], 2023.10,
-
Learning Multiplex Embeddings on Text-rich Networks with One Text Encoder.
preprint
Bowen Jin, Wentao Zhang, Yu Zhang, Yu Meng, Han Zhao, Jiawei Han. [PDF][Code], 2023.10,
-
Disentangled Representation Learning with Large Language Models for Text-Attributed Graphs.
preprint
Yijian Qin, Xin Wang, Ziwei Zhang, Wenwu Zhu. [PDF], 2023.10,
-
ReasoningLM: Enabling Structural Subgraph Reasoning in Pre-trained Language Models for Question Answering over Knowledge Graph.
EMNLP23
Jinhao Jiang, Kun Zhou, Wayne Xin Zhao, Yaliang Li, Ji-Rong Wen. [PDF], 2023.12,
-
Chain of History: Learning and Forecasting with LLMs for Temporal Knowledge Graph Completion.
preprint
Ruilin Luo, Tianle Gu, Haoling Li, Junzhe Li, Zicheng Lin, Jiayi Li, Yujiu Yang. [PDF] [Code], 2024.1,
-
Scalable Link Prediction on Large-Scale Heterogeneous Graphs with Large Language Models.
preprint
Baolong Bi, Shenghua Liu, Yiwei Wang, Lingrui Mei, Xueqi Chen. [PDF], 2024.1,
-
Similarity-based Neighbor Selection for Graph LLMs.
preprint
Rui Li, Jiwei Li, Jiawei Han, Guoyin Wang. [PDF], 2024.2,
-
Let Your Graph Do the Talking: Encoding Structured Data for LLMs.
preprint
Bryan Perozzi, Bahare Fatemi, Dustin Zelle, Anton Tsitsulin, Mehran Kazemi, Rami Al-Rfou, Jonathan Halcrow. [PDF], 2024.2,
-
InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignment.
preprint
Jianing Wang, Junda Wu, Yupeng Hou, Yao Liu, Ming Gao, Julian McAuley. [PDF], 2024.2,
-
Text Generation from Knowledge Graphs with Graph Transformers.
NAACL 2019
Rik Koncel-Kedziorski, Dhanush Bekal, Yi Luan, Mirella Lapata, Hannaneh Hajishirzi. [PDF] [Code], 2019.4,
-
GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph.
NeurIPs 2021
Junhan Yang, Zheng Liu, Shitao Xiao, Chaozhuo Li, Defu Lian, Sanjay Agrawal, Amit Singh, Guangzhong Sun, Xing Xie. [PDF][Code], 2021.5,
-
GreaseLM: Graph Reasoning Enhanced Language Models for Question Answering.
ICLR 2022
Xikun Zhang, Antoine Bosselut, Michihiro Yasunaga, Hongyu Ren, Percy Liang, Christopher D Manning and Jure Leskovec. [PDF] [Code], 2022.1,
-
Heterformer: Transformer-based Deep Node Representation Learning on Heterogeneous Text-Rich Networks.
KDD 2023
Bowen Jin, Yu Zhang, Qi Zhu, Jiawei Han. [PDF][Code], 2022.5,
-
Hidden Schema Networks.
preprint
Ramsés J. Sánchez, Lukas Conrads, Pascal Welke, Kostadin Cvejoski, César Ojeda. [PDF], 2022.7,
-
DRAGON: Deep Bidirectional Language-Knowledge Graph Pretraining.
NeurIPs 2022
Michihiro Yasunaga, Antoine Bosselut, Hongyu Ren, Xikun Zhang, Christopher D. Manning, Percy Liang, Jure Leskovec. [PDF][Code], 2022.10,
-
Edgeformers: Graph-Empowered Transformers for Representation Learning on Textual-Edge Networks.
ICLR 2023
Bowen Jin, Yu Zhang, Yu Meng, Jiawei Han. [PDF][Code], 2023.1,
-
Patton: Language Model Pretraining on Text-rich Networks.
ACL 2023
Bowen Jin, Wentao Zhang, Yu Zhang, Yu Meng, Xinyang Zhang, Qi Zhu, Jiawei Han. [PDF][Code], 2023.5,
-
Graph Language Models.
preprint
Moritz Plenz, Anette Frank. [PDF], 2024.1,
-
Efficient Tuning and Inference for Large Language Models on Textual Graphs.
preprint
Yun Zhu, Yaoke Wang, Haizhou Shi, Siliang Tang. [PDF], 2024.1,
-
HyperBERT: Mixing Hypergraph-Aware Layers with Language Models for Node Classification on Text-Attributed Hypergraphs.
preprint
Adrián Bazaga, Pietro Liò, Gos Micklem. [PDF], 2024.2,
-
Explaining Relationships Between Scientific Documents.
ACL 2021
Kelvin Luu, Xinyi Wu, Rik Koncel-Kedziorski, Kyle Lo, Isabel Cachola, Noah A. Smith. [PDF], 2020.2,
-
SPECTER: Document-level Representation Learning using Citation-informed Transformers.
ACL 2020
Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, Daniel S. Weld. [PDF], 2020.4,
-
Pre-training for Ad-hoc Retrieval: Hyperlink is Also You Need.
CIKM 2021
Zhengyi Ma, Zhicheng Dou, Wei Xu, Xinyu Zhang, Hao Jiang, Zhao Cao, Ji-Rong Wen. [PDF] [Code], 2021.8,
-
Neighborhood Contrastive Learning for Scientific Document Representations with Citation Embeddings.
EMNLP 2022
Malte Ostendorff, Nils Rethmeier, Isabelle Augenstein, Bela Gipp, Georg Rehm. [PDF][Code], 2022.2,
-
Metadata-Induced Contrastive Learning for Zero-Shot Multi-Label Text Classification.
WWW 2022
Yu Zhang, Zhihong Shen, Chieh-Han Wu, Boya Xie, Junheng Hao, Ye-Yi Wang, Kuansan Wang, Jiawei Han. [PDF][Code], 2022.2,
-
LinkBERT: Pretraining Language Models with Document Links.
ACL 2022
Michihiro Yasunaga, Jure Leskovec, Percy Liang. [PDF][Code], 2022.3,
-
E2EG: End-to-End Node Classification Using Graph Topology and Text-based Node Attributes.
ICDM 2023
Tu Anh Dinh, Jeroen den Boef, Joran Cornelisse, Paul Groth. [PDF], 2022.8,
-
TwHIN-BERT: A Socially-Enriched Pre-trained Language Model for Multilingual Tweet Representations at Twitter.
KDD 2023
Xinyang Zhang, Yury Malkov, Omar Florez, Serim Park, Brian McWilliams, Jiawei Han, Ahmed El-Kishky. [PDF] [Code], 2022.9,
-
TouchUp-G: Improving Feature Representation through Graph-Centric Finetuning.
preprint
Jing Zhu, Xiang Song, Vassilis N. Ioannidis, Danai Koutra, Christos Faloutsos. [PDF], 2023.9,
-
Wikiformer: Pre-training with Structured Information of Wikipedia for Ad-hoc Retrieval.
preprint
Weihang Su, Qingyao Ai, Xiangsheng Li, Jia Chen, Yiqun Liu, Xiaolong Wu, Shengluan Hou. [PDF], 2023.12,
-
WalkLM: A Uniform Language Model Fine-tuning Framework for Attributed Graph Embedding.
NeurIPs 2023
Yanchao Tan, Zihao Zhou, Hang Lv, Weiming Liu, Carl Yang. [PDF], 2023.12,
-
GLaM: Fine-Tuning Large Language Models for Domain Knowledge Graph Alignment via Neighborhood Partitioning and Generative Subgraph Encoding.
AAAI 2024
Stefan Dernbach, Khushbu Agarwal, Alejandro Zuniga, Michael Henry, Sutanay Choudhury. [PDF], 2024.2,
-
GNN-LM: Language Modeling based on Global Contexts via GNN.
ICLR 2022
Yuxian Meng, Shi Zong, Xiaoya Li, Xiaofei Sun, Tianwei Zhang, Fei Wu, Jiwei Li. [PDF] [Code], 2021.10,
-
Node Feature Extraction by Self-Supervised Multi-Scale Neighborhood Prediction.
ICLR 2022
Eli Chien, Wei-Cheng Chang, Cho-Jui Hsieh, Hsiang-Fu Yu, Jiong Zhang, Olgica Milenkovic, Inderjit S Dhillon. [PDF][Code], 2021.11,
-
TextGNN: Improving Text Encoder via Graph Neural Network in Sponsored Search.
WWW 2021
Jason Yue Zhu, Yanling Cui, Yuming Liu, Hao Sun, Xue Li, Markus Pelger, Tianqi Yang, Liangjie Zhang, Ruofei Zhang, Huasha Zhao. [PDF] [Code], 2022.1
-
AdsGNN: Behavior-Graph Augmented Relevance Modeling in Sponsored Search.
SIGIR 2021
Chaozhuo Li, Bochen Pang, Yuming Liu, Hao Sun, Zheng Liu, Xing Xie, Tianqi Yang, Yanling Cui, Liangjie Zhang, Qi Zhang. [PDF] [Code], 2022.4
-
Efficient and effective training of language and graph neural network models.
AAAI 2023
Vassilis N Ioannidis, Xiang Song, Da Zheng, Houyu Zhang, Jun Ma, Yi Xu, Belinda Zeng, Trishul Chilimbi, George Karypis. [PDF], 2022.6,
-
Graph-Aware Language Model Pre-Training on a Large Graph Corpus Can Help Multiple Graph Applications.
KDD 2023
Han Xie, Da Zheng, Jun Ma, Houyu Zhang, Vassilis N. Ioannidis, Xiang Song, Qing Ping, Sheng Wang, Carl Yang, Yi Xu, Belinda Zeng, Trishul Chilimbi. [PDF], 2023.6,
-
Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs.
preprint
Zhikai Chen, Haitao Mao, Hang Li, Wei Jin, Hongzhi Wen, Xiaochi Wei, Shuaiqiang Wang, Dawei Yin, Wenqi Fan, Hui Liu, Jiliang Tang. [PDF] [Code], 2023.7,
-
SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learning.
preprint
Keyu Duan, Qian Liu, Tat-Seng Chua, Shuicheng Yan, Wei Tsang Ooi, Qizhe Xie, Junxian He. [PDF] [Code], 2023.8,
-
Explanations as Features: LLM-Based Features for Text-Attributed Graphs.
preprint
Xiaoxin He, Xavier Bresson, Thomas Laurent, Adam Perold, Yann LeCun, Bryan Hooi. [PDF] [Code], 2023.5,
-
Label-free Node Classification on Graphs with Large Language Models (LLMS).
preprint
Zhikai Chen, Haitao Mao, Hongzhi Wen, Haoyu Han, Wei Jin, Haiyang Zhang, Hui Liu, Jiliang Tang. [PDF], 2023.9,
-
Empower Text-Attributed Graphs Learning with Large Language Models (LLMs).
preprint
Jianxiang Yu, Yuxiang Ren, Chenghua Gong, Jiaqi Tan, Xiang Li, Xuecang Zhang. [PDF], 2023.10,
-
Large Language Models as Topological Structure Enhancers for Text-Attributed Graphs.
preprint
Shengyin Sun, Yuxiang Ren, Chen Ma, Xuecang Zhang. [PDF], 2023.11,
-
A Comprehensive Study on Text-attributed Graphs: Benchmarking and Rethinking.
NeurIPS 2023
Hao Yan, Chaozhuo Li, Ruosong Long, Chao Yan, Jianan Zhao, Wenwen Zhuang, Jun Yin, Peiyan Zhang, Weihao Han, Hao Sun, Weiwei Deng, Qi Zhang, Lichao Sun, Xing Xie, Senzhang Wang [PDF] [Code], 2023.11,
-
Distilling Event Sequence Knowledge From Large Language Models.
preprint
Somin Wadhwa, Oktie Hassanzadeh, Debarun Bhattacharjya, Ken Barker, Jian Ni. [PDF], 2024.1,
-
Train Your Own GNN Teacher: Graph-Aware Distillation on Textual Graphs.
PKDD 2023
C. Mavromatis, V. N. Ioannidis, S. Wang, D. Zheng, S. Adeshina, J. Ma, H. Zhao, C. Faloutsos, G. Karypis. [PDF], 2023.4,
-
Efficient Large Language Models Fine-Tuning On Graphs.
preprint
Rui Xue, Xipeng Shen, Ruozhou Yu, Xiaorui Liu. [PDF], 2023.12,
-
Distilling Large Language Models for Text-Attributed Graph Learning.
preprint
Bo Pan, Zheng Zhang, Yifei Zhang, Yuntong Hu, Liang Zhao. [PDF], 2024.2,
-
Parameter-Efficient Tuning Large Language Models for Graph Representation Learning.
preprint
Qi Zhu, Da Zheng, Xiang Song, Shichang Zhang, Bowen Jin, Yizhou Sun, George Karypis. [PDF], 2024.4,
-
Minimally-Supervised Structure-Rich Text Categorization via Learning on Text-Rich Networks.
WWW 2021
Xinyang Zhang, Chenwei Zhang, Luna Xin Dong, Jingbo Shang, Jiawei Han. [PDF] [Code], 2021.2,
-
Learning on Large-scale Text-attributed graphs via variational inference.
ICLR 2023
Jianan Zhao, Meng Qu, Chaozhuo Li, Hao Yan, Qian Liu, Rui Li, Xing Xie, Jian Tang. [PDF][Code], 2023.1,
-
ConGraT: Self-Supervised Contrastive Pretraining for Joint Graph and Text Embeddings.
preprint
William Brannon, Suyash Fulay, Hang Jiang, Wonjune Kang, Brandon Roy, Jad Kabbara, Deb Roy. [PDF] [Code], 2023.5,
-
Augmenting Low-Resource Text Classification with Graph-Grounded Pre-training and Prompting.
SIGIR 2023
Zhihao Wen, Yuan Fang. [PDF], 2023.5,
-
GRENADE: Graph-Centric Language Model for Self-Supervised Representation Learning on Text-Attributed Graphs.
EMNLP 2023
-
Pretraining Language Models with Text-Attributed Heterogeneous Graphs.
preprint
Tao Zou, Le Yu, Yifei HUANG, Leilei Sun, Bowen Du. [PDF] [Code], 2023.10,
Table 8 in our survey paper Large Language Models on Graphs: A Comprehensive Survey.
-
SMILES-BERT: Large Scale Unsupervised Pre-Training for Molecular Property Prediction
BCB 19
Sheng Wang , Yuzhi Guo , Yuhong Wang , Hongmao Sun , Junzhou Huang [PDF] [Code], 2019.09,
-
MolGPT: Molecular Generation Using a Transformer-Decoder Model.
Journal of Chemical Information and Modeling
Viraj Bagal, Rishal Aggarwal, P. K. Vinod, and U. Deva Priyakumar [PDF] [Code], 2021.10,
,
-
A Deep-learning System Bridging Molecule Structure and Biomedical Text with Comprehension Comparable to Human Professionals.
Nature Communications
Zheni Zeng, Yuan Yao, Zhiyuan Liu, Maosong Sun [PDF] [Code], 2022.02,
-
Chemformer: a pre-trained transformer for computational chemistry
Machine Learning: Science and Technology
Ross Irwin, Spyridon Dimitriadis, Jiazhen He and Esben Jannik Bjerrum [PDF] [Code], 2022.02,
-
Large-Scale Distributed Training of Transformers for Chemical Fingerprinting.
Journal of Chemical Information and Modeling
-
Galactica: A Large Language Model for Science
Preprint
Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, Robert Stojnic [PDF] [Code], 2022.11,
-
Translation between Molecules and Natural Language.
EMNLP 2022
Carl Edwards, Tuan Lai, Kevin Ros, Garrett Honke, Kyunghyun Cho, Heng Ji. [PDF] [Code], 2022.12,
-
Unifying Molecular and Textual Representations via Multi-task Language Modelling
ICML 2023
Dimitrios Christofidellis, Giorgio Giannone, Jannis Born, Ole Winther, Teodoro Laino, Matteo Manica [PDF] [Code], 2023.05,
-
What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks.
NeurIPS 2023
Taicheng Guo, Kehan Guo, Bozhao Nan, Zhenwen Liang, Zhichun Guo, Nitesh V. Chawla, Olaf Wiest, Xiangliang Zhang [PDF] [Code], 2023.9,
-
MolXPT: Wrapping Molecules with Text for Generative Pre-training.
ACL 2023
Zequn Liu, Wei Zhang, Yingce Xia, Lijun Wu, Shufang Xie, Tao Qin, Ming Zhang, Tie-Yan Liu. [PDF], 2023.05,
-
Interactive Molecular Discovery with Natural Language
preprint
Zheni Zeng, Bangchen Yin, Shipeng Wang, Jiarui Liu, Cheng Yang, Haishen Yao, Xingzhi Sun, Maosong Sun, Guotong Xie, Zhiyuan Liu. [PDF] [Code], 2023.06,
-
Empowering Molecule Discovery for Molecule-Caption Translation with Large Language Models: A ChatGPT Perspective
preprint
Jiatong Li, Yunqing Liu, Wenqi Fan, Xiao-Yong Wei, Hui Liu, Jiliang Tang, Qing Li. [PDF], 2023.06, [Code],
-
Regression Transformer enables concurrent sequence regression and generation for molecular language modelling
Nature Machine Intelligence
-
Can Large Language Models Empower Molecular Property Prediction?
preprint
Chen Qian, Huayi Tang, Zhirui Yang, Hong Liang, Yong Liu. [PDF] [Code], 2023.7,
-
Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Models
preprint
Yin Fang, Xiaozhuan Liang, Ningyu Zhang, Kangwei Liu, Rui Huang, Zhuo Chen, Xiaohui Fan, Huajun Chen [PDF], 2023.07,
-
GPT-MolBERTa: GPT Molecular Features Language Model for molecular property prediction.
preprint
Balaji, Suryanarayanan, Rishikesh Magar, and Yayati Jadhav. [PDF], 2023.10,
-
The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4.
preprint
Microsoft Research AI4Science, Microsoft Azure Quantum [PDF], 2023.11,
-
Towards 3D Molecule-Text Interpretation in Language Models
ICLR 2024
Sihang Li, Zhiyuan Liu, Yanchen Luo, Xiang Wang, Xiangnan He, Kenji Kawaguchi, Tat-Seng Chua, Qi Tian. [PDF], 2024.1,
-
MolTC: Towards Molecular Relational Modeling In Language Models
preprint
Junfeng Fang, Shuai Zhang, Chang Wu, Zhengyi Yang, Zhiyuan Liu, Sihang Li, Kun Wang, Wenjie Du, Xiang Wang. [PDF], 2024.2,
-
Large Language Models are In-Context Molecule Learners
preprint
Jiatong Li, Wei Liu, Zhihao Ding, Wenqi Fan, Yuqiang Li, Qing Li. [PDF], 2024.3,
-
Text2Mol: Cross-Modal Molecule Retrieval with Natural Language Queries.
EMNLP 2021
Carl Edwards, ChengXiang Zhai, Heng Ji. [PDF] [Code], 2021.10,
-
GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learning.
preprint
Haiteng Zhao, Shengchao Liu, Chang Ma, Hannan Xu, Jie Fu, Zhi-Hong Deng, Lingpeng Kong, Qi Liu. [PDF] [Code], 2023.6,
-
Prot2Text: Multimodal Protein's Function Generation with GNNs and Transformers
preprint
Hadi Abdine, Michail Chatzianastasis, Costas Bouyioukos, Michalis Vazirgiannis [PDF], 2023.07,
-
ReLM: Leveraging Language Models for Enhanced Chemical Reaction Prediction.
EMNLP 2023
Yaorui Shi, An Zhang, Enzhi Zhang, Zhiyuan Liu, Xiang Wang. [PDF] [Code], 2023.10,
-
Biomedical knowledge graph-enhanced prompt generation for large language models
preprint
Karthik Soman, Peter W Rose, John H Morris, Rabia E Akbas, Brett Smith, Braian Peetoom, Catalina Villouta-Reyes, Gabriel Cerono, Yongmei Shi, Angela Rizk-Jackson, Sharat Israni, Charlotte A Nelson, Sui Huang, Sergio E Baranzini [PDF] [Code], 2023.11,
-
L+M-24: Building a Dataset for Language + Molecules @ ACL 2024
preprint
Carl Edwards, Qingyun Wang, Lawrence Zhao, Heng Ji [PDF] [Code], 2024.2,
-
A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Language.
preprint
Bing Su, Dazhao Du, Zhao Yang, Yujie Zhou, Jiangmeng Li, Anyi Rao, Hao Sun, Zhiwu Lu, Ji-Rong Wen [PDF] [Code], 2022.09,
-
Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editing.
preprint
Shengchao Liu, Weili Nie, Chengpeng Wang, Jiarui Lu, Zhuoran Qiao, Ling Liu, Jian Tang, Chaowei Xiao, Anima Anandkumar [PDF], 2022.10,
-
Extracting Molecular Properties from Natural Language with Multimodal Contrastive Learning
ICML 2023 Workshop on Computational Biology
Romain Lacombe, Andrew Gaut, Jeff He, David Lüdeke, Kateryna Pistunova [PDF], 2023.07,
-
ProtST: Multi-Modality Learning of Protein Sequences and Biomedical Texts.
ICML 2023
Minghao Xu, Xinyu Yuan, Santiago Miret, Jian Tang. [PDF] [Code], 2023.1,
-
Enhancing Activity Prediction Models in Drug Discovery with the Ability to Understand Human Language.
ICML 2023
Philipp Seidl, Andreu Vall, Sepp Hochreiter, Günter Klambauer. [PDF] [Code], 2023.5,
-
MolFM: A multimodal molecular foundation model.
Preprint
Yizhen Luo, Kai Yang, Massimo Hong, Xing Yi Liu, Zaiqing Nie. [PDF] [Code], 2023.7,
-
GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Text
Preprint
Pengfei Liu, Yiming Ren, Zhixiang Ren [PDF], 2023.08,
-
MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter.
EMNLP 2023
Zhiyuan Liu, Sihang Li, Yanchen Luo, Hao Fei, Yixin Cao, Kenji Kawaguchi, Xiang Wang, Tat-Seng Chua. [PDF] [Code], 2023.10,
-
CHEMREASONER: Heuristic Search over a Large Language Model's Knowledge Space using Quantum-Chemical Feedback.
preprint
Henry W. Sprueill, Carl Edwards, Khushbu Agarwal, Mariefel V. Olarte, Udishnu Sanyal, Conrad Johnston, Hongbin Liu, Heng Ji, Sutanay Choudhury. [PDF], 2024.2,
Contributions to this repository are welcome!
If you find any error or have relevant resources, feel free to open an issue or a pull request.
Paper format:
1. **[paper title].** `[]`
*[authors].* [[PDF]([pdf link])] [[Code]([code link])], published time, ![](https://img.shields.io/badge/[architecture]-blue) ![](https://img.shields.io/badge/[size]-red)
Please cite the following paper if you find the resource helpful for your research.
@article{jin@llmgraph,
title={Large Language Models on Graphs: A Comprehensive Survey},
author={Jin, Bowen and Liu, Gang and Han, Chi and Jiang, Meng and Ji, Heng and Han, Jiawei},
journal={arXiv preprint arXiv:2312.02783},
year={2023}
}