HyperU-GCN

This is a PyTorch implementation of the HyperU-GCN model in our following paper:

Xueying Yang, Jiamian Wang, Xujiang Zhao, Sheng Li and Zhiqiang Tao, "Calibrate Automated Graph Neural Network via Hyperparameter Uncertainty", CIKM 2022.

Introduction

Automated graph learning has drawn widespread research attention due to its great potential to reduce human efforts when dealing with graph data, among which hyperparameter optimization (HPO) is one of the mainstream directions with promising progress. However, how to obtain reliable and trustworthy prediction results with automated graph neural networks (GNN) is still quite underexplored. To this end, we investigate automated GNN calibration by marrying uncertainty estimation to the HPO problem. We propose a hyperparameter uncertainty-induced graph convolutional network (HyperU-GCN) with a bilevel formulation, where the upper-level problem explicitly reasons uncertainties by developing a probabilistic hypernetworks through a variational Bayesian lens, while the lower-level problem learns how the GCN weights respond to a hyperparameter distribution. By squeezing model uncertainty into the hyperparameter space, the proposed HyperU-GCN could achieve calibrated predictions in a similar way to Bayesian model averaging over hyperparameters. Extensive experimental results on six public datasets were provided in terms of node classification accuracy and expected calibration error (ECE), demonstrating the effectiveness of our approach compared with several state-of-the-art uncertainty-aware and calibrated GCN methods.

Installation

Clone the repo:

git clone https://github.com/xyang2316/HyperU-GCN
cd HyperU-GCN

Install the following dependencies, including:

Python 3.9.7
PyTorch 1.11
numpy 1.21.2
scipy 1.7.3
sklearn 1.0.2
networkx 2.6.3

Datasets

Cora, Citeseer, Pubmed,
Coauthor Physics, Amazon Computer, and Amazon Photo

Run the model:

Run HyperU_GCN_train.py for different datasets:

Cora

python HyperU_GCN_train.py --dataset=cora --start_dropedge=0.3 --start_dropout=0.1 --start_weightdecay=3e-4 --total_epochs=400 --train_lr=5e-4

Citeseer

python HyperU_GCN_train.py --dataset=citeseer --start_dropedge=0.3 --start_dropout=0.08 --start_weightdecay=3e-4 --total_epochs=400 --train_lr=5e-4

Pubmed

python HyperU_GCN_train.py --dataset=pubmed --start_dropedge=0.8 --start_dropout=0.05 --start_weightdecay=5e-4 --total_epochs=400 --train_lr=5e-3

Physics

python HyperU_GCN_train.py --dataset=ms_academic_phy --start_dropedge=0.3 --start_dropout=0.1 --start_weightdecay=5e-5 --total_epochs=600 --train_lr=5e-4

Computers

python HyperU_GCN_train.py --dataset=amazon_electronics_computers --start_dropedge=0.3 --start_dropout=0.05 --start_weightdecay=5e-4 --total_epochs=600 --train_lr=5e-4

Photo

python HyperU_GCN_train.py --dataset=amazon_electronics_photo --start_dropedge=0.3 --start_dropout=0.1 --start_weightdecay=5e-5 --total_epochs=600 --train_lr=5e-4

Other parameters stays as the default values.

Results

The expeiment results in our paper:

Citation

If you find the code helpful in your resarch or work, please cite our paper:

@inproceedings{yang2022calibrate,
  title={Calibrate Automated Graph Neural Network via Hyperparameter Uncertainty},
  author={Yang, Xueying and Wang, Jiamian and Zhao, Xujiang and Li, Sheng and Tao, Zhiqiang},
  booktitle={Proceedings of the 31st ACM International Conference on Information \& Knowledge Management},
  pages={4640--4644},
  year={2022}
}