HyperMODEST

HyperMODEST: Self-Supervised 3D Object Detection with Confidence Score Filtering
Jenny Xu, Steven L. Waslander.
2023 20th Conference on Computer and Robot Vision (CRV), Montreal, QC, Canada, 2023.
https://arxiv.org/abs/2304.14446

This repository is based off MODEST.

Overview

Abstract
Changelog
Model Zoo
Installation
Getting Started
License
Acknowledgement

Abstract

Current LiDAR-based 3D object detectors for autonomous driving are almost entirely trained on human-annotated data collected in specific geographical domains with specific sensor setups, making it difficult to adapt to a different domain. MODEST is the first work to train 3D object detectors without any labels. Our work, HyperMODEST, proposes a universal method implemented on top of MODEST that can largely accelerate the self-training process and does not require tuning on a specific dataset. We filter intermediate pseudo-labels used for data augmentation with low confidence scores. On the nuScenes dataset, we observe a significant improvement of 1.6% in AP_BEV in 0-80m range at IoU=0.25 and an improvement of 1.7% in AP_BEV in 0-80m range at IoU=0.5 while only using one-fifth of the training time in the original approach by MODEST. On the Lyft dataset, we also observe an improvement over the baseline during the first round of iterative self-training. We explore the trade-off between high precision and high recall in the early stage of the self-training process by comparing our proposed method with two other score filtering methods: confidence score filtering for pseudo-labels with and without static label retention.

Changelog

[2023-03-12] HyperMODEST v0.1.0 is released.

Model Zoo

nuScenes experiments

Model	ST rounds	Checkpoint	Config file
PointRCNN	1	link	cfg
PointRCNN	2	link	cfg
PointRCNN	3	link	cfg
PointRCNN	4	link	cfg

Lyft experiments

Model	ST rounds	Checkpoint	Config file
PointRCNN	1	link	cfg

Installation

conda create --name modest python=3.8
conda activate modest
conda install pytorch=1.9.0 torchvision torchaudio cudatoolkit=11.1 -c pytorch -c nvidia
pip install opencv-python matplotlib wandb scipy tqdm easydict scikit-learn pyquaternion pillow==8.3.2
# for managing experiments
pip install hydra-core --upgrade
pip install hydra_colorlog --upgrade
pip install rich

cd generate_cluster_mask/utils/iou3d_nms
python setup.py install

for OpenPCDet, follow downstream/OpenPCDet/docs/INSTALL.md to install.

Getting Started

Data Pre-processing

Please refer to data_preprocessing/lyft/LYFT_PREPROCESSING.md and data_preprocessing/nuscenes/NUSCENES_PREPROCESSING.md.

Training

1. Generate Seed Labels

Lyft data

cd generate_cluster_mask
# generate pp score
python pre_compute_pp_score.py data_root=$(pwd)/../downstream/OpenPCDet/data/lyft/training
# generate seed labels
python generate_mask.py data_root=$(pwd)/../downstream/OpenPCDet/data/lyft/training
python generate_label_files.py data_root=$(pwd)/../downstream/OpenPCDet/data/lyft/training

nuScenes data

cd generate_cluster_mask
# generate pp score
python pre_compute_pp_score.py data_paths=nusc.yaml data_root=NUSCENES_KITTI_FORMAT_20HZ/training \
  nusc=True
# generate seed labels
python generate_mask.py data_paths=nusc.yaml data_root=$(pwd)/../downstream/OpenPCDet/data/nuscenes_boston/training plane_estimate.max_hs=-1.3
python generate_label_files.py data_paths=nusc.yaml data_root=$(pwd)/../downstream/OpenPCDet/data/nuscenes_boston/training image_shape="[900, 1600]"

2. Run 0-th Round Training with seed labels

Lyft (default PRCNN model)

# set the flag "score_filter" to False on line 597 in 
# downstream/OpenPCDet/pcdet/datasets/kitti/kitti_dataset.py.
bash scripts/seed_training_lyft.sh

nuScenes (default PRCNN model)

# set the flag "score_filter" to False on line 597 in 
# downstream/OpenPCDet/pcdet/datasets/kitti/kitti_dataset.py.
bash scripts/seed_training_nuscenes.sh

3. Self-training (HyperMODEST: Filter-Data-Augmentation)