ZhangYY12345/SGNet

SGNet: Structure Guided Network via Gradient-Frequency Awareness for Depth Map Super-Resolution (AAAI-2024)

SGNet: Structure Guided Network via Gradient-Frequency Awareness for Depth Map Super-Resolution

Zhengxue Wang, Zhiqiang Yan✉, Jian Yang✉

PCA Lab, Nanjing University of Science and Technology, China

This repository is an official PyTorch implementation of our SGNet (AAAI 2024).

Dependencies

Python==3.11.5
PyTorch==2.1.0
numpy==1.23.5 
torchvision==0.16.0
scipy==1.11.3
thop==0.1.1.post2209072238
Pillow==10.0.1
tqdm==4.65.0

Models

All pretrained models can be found here. Please note that some variable names in the initial pretrained .pth files are not consistent with those in the latest code. Therefore, we have reuploaded the new .pth files for completeness, named xxx_R.pth.

Datasets

NYU-v2

RGB-D-D

Lu & Middlebury

Train on synthetic NYU-v2

x4 DSR

python train.py --scale 4 --num_feats 48

x8 DSR

python train.py --scale 8 --num_feats 40

x16 DSR

python train.py --scale 16 --num_feats 40

Train on real-world RGB-D-D

python train.py --scale 4 --num_feats 24

Test on synthetic datasets

x4 DSR

python test.py --scale 4 --num_feats 48

x8 DSR

python test.py --scale 8 --num_feats 40

x16 DSR

python test.py --scale 16 --num_feats 40

Test on real-world RGB-D-D

python test.py --scale 4 --num_feats 24

Experiments

Visual comparison

Train & test on real-world RGB-D-D:

Train & test on synthetic NYU-v2 (x16):

Train on NYU-v2, test on RGB-D-D (x16):

Acknowledgements

We thank all reviewers for their professional and instructive suggestions.

We thank these repos sharing their codes: DKN and SUFT.

Citation

If our method proves to be of any assistance, please consider citing:

@article{wang2023sgnet,
  title={SGNet: Structure Guided Network via Gradient-Frequency Awareness for Depth Map Super-Resolution},
  author={Wang, Zhengxu and Yan, Zhiqiang and Yang, Jian},
  journal={arXiv preprint arXiv:2312.05799},
  year={2023}
}