RFNet: Real-time Fusion Network for RGB-D Semantic Segmentation Incorporating Unexpected Obstacle Detection of Road-driving Images
This repository is a Pytorch implementation for
Sun, Lei, et al. "Real-time Fusion Network for RGB-D Semantic Segmentation Incorporating Unexpected Obstacle Detection for Road-driving Images." arXiv preprint arXiv:2002.10570 (2020).
If you want to use this code in your research, please cite the paper.
Requirement
Python 3.6
Pytorch 1.1
Torchvision 0.3
Opencv 3.3.1
Datasets
Get dataset from Cityscapes, and from Lost and Found.
If you want to use multi-dataset training, mix two datasets and the directory structure should be like this:
├─disparity
│ ├─test
│ │ ├─berlin
│ │ ├─bielefeld
│ │ ├─bonn
│ │ ├─...
│ │ └─munich
│ ├─train
│ │ ├─01_Hanns_Klemm_Str_45
│ │ ├─03_Hanns_Klemm_Str_19
│ │ ├─...
│ │ └─zurich
│ └─val
│ ├─02_Hanns_Klemm_Str_44
│ ├─04_Maurener_Weg_8
│ ├─05_Schafgasse_1
│ ├─...
│ └─munster
├─gtFine
│ ├─train
│ │ ├─01_Hanns_Klemm_Str_45
│ │ ├─03_Hanns_Klemm_Str_19
│ │ ├─...
│ │ └─zurich
│ └─val
│ ├─02_Hanns_Klemm_Str_44
│ ├─04_Maurener_Weg_8
│ ├─05_Schafgasse_1
│ ├─...
│ └─munster
└─leftImg8bit
├─test
│ ├─berlin
│ ├─bielefeld
│ ├─bonn
│ ├─...
│ └─munich
├─train
│ ├─01_Hanns_Klemm_Str_45
│ ├─03_Hanns_Klemm_Str_19
│ ├─...
│ └─zurich
└─val
├─02_Hanns_Klemm_Str_44
├─04_Maurener_Weg_8
├─05_Schafgasse_1
├─...
└─munster
Pretrained weights
BaiduNetdisk
Download weights for Cityscapes, password: 4lts
Download weights for Multi-dataset: Cityscapes and Lost and Found, password: t1mh
Google Drive
Download weights for Cityscapes
Download weights for Multi-dataset: Cityscapes and Lost and Found
Training
Edit path to your dataset in mypath.py
.
python train.py --depth --lr 1e-4 --weight-decay 2.5e-5 --workers 4 --epochs 200 --batch-size 8 --val-batch-size 3 --gpu-ids 0 --checkname test --eval-interval 2 --dataset citylostfound --loss-type ce --use-balanced-weights
Evaluation
python eval.py --dataset citylostfound --weight-path your/path/to/weight/ --depth
Note: The code is partially based Swiftnet and DeepLab v3+.