/FaPN

[ICCV 2021] FaPN: Feature-aligned Pyramid Network for Dense Image Prediction

Primary LanguageC++Apache License 2.0Apache-2.0

FaPN: Feature-aligned Pyramid Network for Dense Image Prediction [arXiv] [Project Page]

@inproceedings{
  huang2021fapn,
  title={{FaPN}: Feature-aligned Pyramid Network for Dense Image Prediction},
  author={Shihua Huang and Zhichao Lu and Ran Cheng and Cheng He},
  booktitle={International Conference on Computer Vision (ICCV)},
  year={2021}
}

Overview

FaPN vs. FPN Before vs. After Alignment

This project provides the official implementation for our ICCV2021 paper "FaPN: Feature-aligned Pyramid Network for Dense Image Prediction" based on Detectron2. FaPN is a simple yet effective top-down pyramidal architecture to generate multi-scale features for dense image prediction. Comprised of a feature alignment module (FAM) and a feature selection module (FSM), FaPN addresses the issue of feature alignment in the original FPN, leading to substaintial improvements on various dense prediction tasks, such as object detection, semantic, instance, panoptic segmentation, etc.

Installation

This project is based on Detectron2, which can be constructed as follows.

Training

To train a model with 8 GPUs, run:

cd /path/to/detectron2/tools
python3 train_net.py --config-file <config.yaml> --num-gpus 8

For example, to launch Faster R-CNN training (1x schedule) with ResNet-50 backbone on 8 GPUs, one should execute:

cd /path/to/detectron2/tools
python3 train_net.py --config-file ../configs\COCO-Detection\faster_rcnn_R_50_FAN_1x.yaml --num-gpus 8

Evaluation

To evaluate a pre-trained model with 8 GPUs, run:

cd /path/to/detectron2/tools
python3 train_net.py --config-file <config.yaml> --num-gpus 8 --eval-only MODEL.WEIGHTS /path/to/model_checkpoint

Results

COCO Object Detection

Faster R-CNN + FaPN:

Name lr
sched
box
AP
box
APs
box
APm
box
APl
download
R50 1x 39.2 24.5 43.3 49.1 model |  log
R101 3x 42.8 27.0 46.2 54.9 model |  log

Cityscapes Semantic Segmentation

PointRend + FaPN:

Name lr
sched
mask
mIoU
mask
i_IoU
mask
IoU_sup
mask
iIoU_sup
download
R50 1x 80.0 61.3 90.6 78.5 model |  log
R101 1x 80.1 62.2 90.8 78.6 model |  log

COCO Instance Segmentation

Mask R-CNN + FaPN:

Name lr
sched
mask
AP
mask
APs
box
AP
box
APs
download
R50 1x 36.4 18.1 39.8 24.3 model |  log
R101 3x 39.4 20.9 43.8 27.4 model |  log

PointRend + FaPN:

Name lr
sched
mask
AP
mask
APs
box
AP
box
APs
download
R50 1x 37.6 18.6 39.4 24.2 model |  log

COCO Panoptic Segmentation

PanopticFPN + FaPN:

Name lr
sched
PQ mask
mIoU
St
PQ
box
AP
Th
PQ
download
R50 1x 41.1 43.4 32.5 38.7 46.9 model |  log
R101 3x 44.2 45.7 35.0 43.0 53.3 model |  log