[CVPR 2022] Eigencontours: Novel Contour Descriptors Based on Low-Rank Approximation (Oral)

Wonhui Park, Dongkwon Jin, and Chang-Su Kim

Official implementation for "Eigencontours: Novel Contour Descriptors Based on Low-Rank Approximation" [paper] [arxiv] [video]

Related work

We wil also present another paper, "Eigenlanes: Data-Driven Lane Descriptors for Structurally Diverse Lanes", accepted to CVPR 2022 [paper] [arxiv] [video] [github]. Congratulations to my eigenbrother!

Requirements

PyTorch >= 1.4.0
CUDA >= 10.0
CuDNN >= 7.6.5
python >= 3.6

Installation

Download repository. We call this directory as ROOT:

$ git clone https://github.com/dnjs3594/Eigencontours.git

Download preprocessed-data in ROOT/Preprocessing/code_v1_COCO:

$ cd ROOT/Preprocessing/code_v1_COCO
$ unzip data.zip

Create conda environment:

$ conda create -n eigencontours python=3.7 anaconda
$ conda activate eigencontours

Install dependencies:

$ conda install pytorch==1.4.0 torchvision==0.5.0 cudatoolkit=10.1 -c pytorch
$ pip install -r requirements.txt

Optional install dependencies: Our framework for instance segmentation is based on mmdetection. Please check mmdetection.md for installation instructions.
Download instance segmentation checkpoint

$ mkdir checkpoints
$ mv [download_root] checkpoints

Directory structure

.                           # ROOT
├── Preprocessing           # directory for data preprocessing
│   ├── code_v1_COCO        # COCO dataset preprocessing
│   ├── code_v1_SBD         # SBD dataset preprocessing (TBD)
│   ├── data                # dataset storage
│   │   ├── COCO            # COCO dataset
│   │   │   ├──train2017
│   │   │   ├──val2017
│   │   │   └──annotations
│   └── ...
├── Instance_segmentation
│   ├── code_v1_COCO 
│   │   ├── configs            # configuration for instance segmentation
│   │   ├── mmdet              # models
│   │   ├── mmdetection        # installed folder
│   │   ├── tools              # execution 
└── └── ...

COCO dataset

Download COCO dataset in `ROOT/Preprocessing/data as directory structure:

Preprocessing (Construct contour descriptors called "Eigencontours".)

Data preprocessing is divided into three steps, which are "encoding", "svd", "convert". Below we describe each step in detail.

In "encoding", star-convex contours are generated and saved to pickle format. you can set the dimensionality of the contours, N = node_num)

$ cd ROOT/Preprocessing/code_v1_COCO/
$ python main.py --mode encoding --node_num N (ex:360) --display False

In "svd", a contour matrix is constucted and the eigencontours are generated by SVD (Singular Value Decomposition). you can get contour descriptors, matrix "U".

$ cd ROOT/Preprocessing/code_v1_COCO/
$ python main.py --mode svd --display False

In "convert", coefficient vectors are generated by calculating between the star-convex contours and the eigencontours. Moreover, the F-scores of descriptors are also saved. you can set the dimensionality of coefficient vectors, M = rank-M approximation)

$ cd ROOT/Preprocessing/code_v1_COCO/
$ python main.py --mode encoding --dim M (ex:36) --display False

Optionally, you can adjust image size and threshold IoU (in "encoding") in ROOT/Preprocessing/code_v1_COCO/options/config.py.

In paper, objects are cropped and centerally allgned for evaluating. If you want to preprocess data in crop version, you replace "datasets.dataset_coco_not_crop" with "dataset.dataset_coco" in ROOT/Preprocessing/code_v1_COCO/libs/prepare.py.

Instance segmentation (PolarMask-based method using "Eigencontours".)[arxiv]

We only train models for ResNet50 backbone and single-scale training and single-gpu environment.

Train

$ cd ROOT/Instance_segmentation/code_v1_COCO/tools
$ python train.py --work_dir <work_dir> --config <config_dir> --data_root <COCO_dir> --resume_from False

You can adjust more details in a config file. And, <work_dir> is a storage directory for checkpoint.

Test

$ cd ROOT/Instance_segmentation/code_v1_COCO/tools
$ python test.py --work_dir <work_dir> --checkpoint <ckpt_dir> --config <config_dir> --data_root <COCO_dir> --show False

Reference

@Inproceedings{
    park2022eigencontours,
    title={Eigencontours: Novel Contour Descriptors Based on Low-Rank Approximation},
    author={Park, Wonhui and Jin, Dongkwon and Kim, Chang-Su},
    booktitle={CVPR},
    year={2022}
}

xingjici/Eigencontours