By Evgenii Razinkov, Iuliia Saveleva, Jiří Matas
This is the original implementation of "ALFA: Agglomerative Late Fusion Algorithm for Object Detection" https://arxiv.org/pdf/1907.06067.pdf.
Image from PASCAL VOC 2007 test set. Bounding boxes and IoU with ground truth: DeNet – red (IoU = 0.75); SSD – green (IoU = 0.77); ALFA – blue (IoU = 0.93). Ground truth bounding box is in white.
| Detector | fps(Hz) | PASCAL VOC 2007 | PASCAL VOC 2012 | ||
|---|---|---|---|---|---|
| mAP-s(%) | mAP(%) | mAP-s(%) | mAP(%) | ||
| Faster R-CNN | 7 | 77.95 | 78.83 | 72.72 | 73.59 |
| SSD | 59 | 79.26 | 80.37 | 72.89 | 74.17 |
| DeNet | 33 | 78.09 | 79.26 | 70.73 | 72.10 |
| SSD + DeNet | |||||
| NMS | 20.3 | 83.12 | 83.53 | 76.80 | 77.37 |
| DBF | 16.9 | 83.29 | 83.88 | 75.74 | 76.38 |
| Fast ALFA | 20.6 | 83.87 | 84.32 | 76.97 | 77.82 |
| ALFA | 18.1 | 84.16 | 84.41 | 77.52 | 77.98 |
| SSD + DeNet + Faster R-CNN | |||||
| NMS | 5.2 | 84.31 | 84.43 | 78.11 | 78.34 |
| DBF | 4.7 | 84.97 | 85.24 | 75.71 | 75.69 |
| Fast ALFA | 5.2 | 85.78 | 85.67 | 79.16 | 79.42 |
| ALFA | 5.0 | 85.90 | 85.72 | 79.41 | 79.47 |
This repository was tested on python 2.7 and 3.5, platforms Linux and Mac OS.
Download this project and unarchive files in "./SSD_Detections", "./DeNet_detections", "./Faster_R-CNN_detections" to run the scripts
Download PASCAL VOC 2007 and PASCAL VOC 2012 dataset and change "path/to/VOC2007 test/VOC2007", and "path/to/VOC2012 test/VOC2012" in bash commands according to location of datasets on your computer.
To draw image from paper:
python ./draw_paper_pic.py \
--alfa_parameters_json="./Algorithm_parameters/ALFA/SSD_DeNet_0.015_single_cross_validation_parameters_2007.json"Results on PASCAL VOC 2007 differ slightly from paper results due to randomness of a cross-validation procedure.
-
Fps values for base detectors were taken from their papers.
-
Evaluate on PASCAL VOC 2007
python ./cross_validate_base_detector.py \
--imagenames_filename="./PASCAL_VOC_files/imagenames_2007_test.txt" \
--pickled_annots_filename="./PASCAL_VOC_files/annots_2007_test.pkl" \
--dataset_dir="path/to/VOC2007 test/VOC2007" \Change --dataset_dir value to real path /VOC2007 test/VOC2007 then type:
--detections_filename="./SSD_detections/SSD_ovthresh_0.015_single_detections_PASCAL_VOC_2007_test.pkl"Change --detections_filename to path from table below.
| Detector | Detections |
|---|---|
| SSD | ./SSD_detections/SSD_ovthresh_0.015_single_detections_PASCAL_VOC_2007_test.pkl |
| DeNet | ./DeNet_detections/DeNet_ovthresh_0.015_single_detections_PASCAL_VOC_2007_test.pkl |
| Faster R-CNN | ./Faster_R-CNN_detections/Faster_R-CNN_ovthresh_0.015_single_detections_PASCAL_VOC_2007_test.pkl |
- Compute fps
python ./validate_ALFA.py \
--imagenames_filename="./PASCAL_VOC_files/imagenames_2007_test.txt" \
--pickled_annots_filename="./PASCAL_VOC_files/annots_2007_test.pkl" \
--dataset_dir="path/to/VOC2007 test/VOC2007" \Change --dataset_dir value to real path /VOC2007 test/VOC2007 then type:
--alfa_parameters_json="./Algorithm_parameters/ALFA/SSD_DeNet_0.05_single_cross_validation_parameters_2007.json"Change last part of the path --alfa_parameters_json to value from table below.
- Evaluate on PASCAL VOC 2007
python ./cross_validate_ALFA.py \
--imagenames_filename="./PASCAL_VOC_files/imagenames_2007_test.txt" \
--pickled_annots_filename="./PASCAL_VOC_files/annots_2007_test.pkl" \
--dataset_dir="path/to/VOC2007 test/VOC2007" \Change --dataset_dir value to real path /VOC2007 test/VOC2007 then type:
--alfa_parameters_json="./Algorithm_parameters/ALFA/SSD_DeNet_0.05_single_cross_validation_parameters_2007.json"Change last part of the path --alfa_parameters_json to value from table below.
| Algorithm | Parameters | |||
|---|---|---|---|---|
| SSD + DeNet, mAP-s(%) | ||||
| Fast ALFA | SSD_DeNet_0.05_single_cross_validation_parameters_2007.json | |||
| ALFA | SSD_DeNet_0.015_single_cross_validation_parameters_2007.json | |||
| SSD + DeNet, mAP(%) | ||||
| Fast ALFA | SSD_DeNet_0.05_multiple_cross_validation_parameters_2007.json | |||
| ALFA | SSD_DeNet_0.015_multiple_cross_validation_parameters_2007.json | |||
| SSD + DeNet + Faster R-CNN, mAP-s(%) | ||||
| Fast ALFA | SSD_DeNet_Faster_R-CNN_0.05_single_cross_validation_parameters_2007.json | |||
| ALFA | SSD_DeNet_Faster_R-CNN_0.015_single_cross_validation_parameters_2007.json | |||
| SSD + DeNet + Faster R-CNN, mAP(%) | ||||
| Fast ALFA | SSD_DeNet_Faster_R-CNN_0.05_multiple_cross_validation_parameters_2007.json | |||
| ALFA | SSD_DeNet_Faster_R-CNN_0.015_multiple_cross_validation_parameters_2007.json | |||
- Compute fps
python ./validate_DBF.py \
--validation_imagenames_filename="./PASCAL_VOC_files/imagenames_2007_test.txt" \
--validation_pickled_annots_filename="./PASCAL_VOC_files/annots_2007_test.pkl" \
--test_imagenames_filename="./PASCAL_VOC_files/imagenames_2007_test.txt" \
--test_pickled_annots_filename="./PASCAL_VOC_files/annots_2007_test.pkl" \
--validation_dataset_dir="path/to/VOC2007 test/VOC2007" \Change --validation_dataset_dir value to real path /VOC2007 test/VOC2007 then type:
--test_dataset_dir="path/to/VOC2007 test/VOC2007" \Change --test_dataset_dir value to real path /VOC2007 test/VOC2007 then type:
--dbf_parameters_json="./Algorithm_parameters/DBF/SSD_DeNet_0.015_single_cross_validation_parameters_2007.json"Change last part of the path --dbf_parameters_json to value from table below.
- Evaluate on PASCAL VOC 2007
python ./cross_validate_DBF.py \
--imagenames_filename="./PASCAL_VOC_files/imagenames_2007_test.txt" \
--pickled_annots_filename="./PASCAL_VOC_files/annots_2007_test.pkl" \
--dataset_dir="path/to/VOC2007 test/VOC2007" \Change --dataset_dir value to real path /VOC2007 test/VOC2007 then type:
--dbf_parameters_json="./Algorithm_parameters/DBF/SSD_DeNet_0.015_single_cross_validation_parameters_2007.json"Change last part of the path --dbf_parameters_json to value from table below.
| Algorithm | Parameters | |||
|---|---|---|---|---|
| SSD + DeNet, mAP-s(%) | ||||
| DBF | SSD_DeNet_0.015_single_cross_validation_parameters_2007.json | |||
| SSD + DeNet, mAP(%) | ||||
| DBF | SSD_DeNet_0.015_multiple_cross_validation_parameters_2007.json | |||
| SSD + DeNet + Faster R-CNN, mAP-s(%) | ||||
| DBF | SSD_DeNet_Faster_R-CNN_0.015_single_cross_validation_parameters_2007.json | |||
| SSD + DeNet + Faster R-CNN, mAP(%) | ||||
| DBF | SSD_DeNet_Faster_R-CNN_0.015_multiple_cross_validation_parameters_2007.json | |||
- Compute fps
python ./validate_NMS.py \
--imagenames_filename="./PASCAL_VOC_files/imagenames_2007_test.txt" \
--pickled_annots_filename="./PASCAL_VOC_files/annots_2007_test.pkl" \
--dataset_dir="path/to/VOC2007 test/VOC2007" \Change --dataset_dir value to real path /VOC2007 test/VOC2007 then type:
--nms_parameters_json="./Algorithm_parameters/NMS/SSD_DeNet_0.015_single_cross_validation_parameters_2007.json"Change last part of the path --nms_parameters_json to value from table below.
- Evaluate on PASCAL VOC 2007
python ./cross_validate_NMS.py \
--imagenames_filename="./PASCAL_VOC_files/imagenames_2007_test.txt" \
--pickled_annots_filename="./PASCAL_VOC_files/annots_2007_test.pkl" \
--dataset_dir="path/to/VOC2007 test/VOC2007" \Change --dataset_dir value to real path /VOC2007 test/VOC2007 then type:
--nms_parameters_json="./Algorithm_parameters/NMS/SSD_DeNet_0.015_single_cross_validation_parameters_2007.json"Change last part of the path --nms_parameters_json to value from table below.
| Algorithm | Parameters | |||
|---|---|---|---|---|
| SSD + DeNet, mAP-s(%) | ||||
| NMS | SSD_DeNet_0.015_single_cross_validation_parameters_2007.json | |||
| SSD + DeNet, mAP(%) | ||||
| NMS | SSD_DeNet_0.015_multiple_cross_validation_parameters_2007.json | |||
| SSD + DeNet + Faster R-CNN, mAP-s(%) | ||||
| NMS | SSD_DeNet_Faster_R-CNN_0.015_single_cross_validation_parameters_2007.json | |||
| SSD + DeNet + Faster R-CNN, mAP(%) | ||||
| NMS | SSD_DeNet_Faster_R-CNN_0.015_multiple_cross_validation_parameters_2007.json | |||
- Convert "detections_path.pkl" to PASCAL VOC 2012 submission format by running:
python ./detections_to_PASCAL_VOC_2012_submission.py \
--detections_filename="detections_path.pkl" \Change --detections_filename value from table below then type:
--submission_folder="path/to/submission_folder"Change --submission_folder value to real path on your computer
| Detector | Detections |
|---|---|
| SSD | ./SSD_detections/SSD_ovthresh_0.015_single_detections_PASCAL_VOC_2012_test.pkl |
| DeNet | ./DeNet_detections/DeNet_ovthresh_0.015_single_detections_PASCAL_VOC_2012_test.pkl |
| Faster R-CNN | ./Faster_R-CNN_detections/Faster_R-CNN_ovthresh_0.015_single_detections_PASCAL_VOC_2012_test.pkl |
-
Archive "path/to/submission_folder/results" as .tar.gz
-
Upload "path/to/submission_folder/results.tar.gz" to PASCAL VOC 2012 evaluation server
- Get ALFA detections running the following command:
python ./validate_ALFA.py \
--imagenames_filename="./PASCAL_VOC_files/imagenames_2012_test.txt" \
--pickled_annots_filename="./PASCAL_VOC_files/annots_2012_test.pkl" \
--dataset_dir="path/to/VOC2012 test/VOC2012" \Change --dataset_dir value to real path to /VOC2012 test/VOC2012 then type:
--alfa_parameters_json="./Algorithm_parameters/ALFA/SSD_DeNet_0.05_single_cross_validation_parameters_2012.json" \Change --alfa_parameters_json to value value from table below then type:
--output_filename="path/to/output_filename.pkl"Change --output_filename to real path on your computer
| Algorithm | Parameters |
|---|---|
| SSD + DeNet, mAP-s(%) | |
| Fast ALFA | SSD_DeNet_0.05_single_cross_validation_parameters_2012.json |
| ALFA | SSD_DeNet_0.015_single_cross_validation_parameters_2012.json |
| SSD + DeNet, mAP(%) | |
| Fast ALFA | SSD_DeNet_0.05_multiple_cross_validation_parameters_2012.json |
| ALFA | SSD_DeNet_0.015_multiple_cross_validation_parameters_2012.json |
| SSD + DeNet + Faster R-CNN, mAP-s(%) | |
| Fast ALFA | SSD_DeNet_Faster_R-CNN_0.05_single_cross_validation_parameters_2012.json |
| ALFA | SSD_DeNet_Faster_R-CNN_0.015_single_cross_validation_parameters_2012.json |
| SSD + DeNet + Faster R-CNN, mAP(%) | |
| Fast ALFA | SSD_DeNet_Faster_R-CNN_0.05_multiple_cross_validation_parameters_2012.json |
| ALFA | SSD_DeNet_Faster_R-CNN_0.015_multiple_cross_validation_parameters_2012.json |
- Convert "path/to/output_filename.pkl" to PASCAL VOC 2012 submission format by running:
python ./detections_to_PASCAL_VOC_2012_submission.py \
--detections_filename="detections_path.pkl" \Change --detections_filename value from table below then type:
--submission_folder="path/to/submission_folder"Change --submission_folder value to real path on your computer
-
Archive "path/to/submission_folder/results" as .tar.gz
-
Upload "path/to/submission_folder/results.tar.gz" to PASCAL VOC 2012 evaluation server
- Get DBF detections running the following command:
python ./validate_DBF.py \
--validation_imagenames_filename="./PASCAL_VOC_files/imagenames_2007_test.txt" \
--validation_pickled_annots_filename="./PASCAL_VOC_files/annots_2007_test.pkl" \
--test_imagenames_filename="./PASCAL_VOC_files/imagenames_2012_test.txt" \
--test_pickled_annots_filename="./PASCAL_VOC_files/annots_2012_test.pkl" \
--validation_dataset_dir="path/to/VOC2007 test/VOC2007" \Change --validation_dataset_dir value to real path to /VOC2007 test/VOC2007 then type:
--test_dataset_dir="path/to/VOC2012 test/VOC2012" \Change --test_dataset_dir value to real path to /VOC2012 test/VOC2012 then type:
--dbf_parameters_json="./Algorithm_parameters/DBF/SSD_DeNet_0.015_single_cross_validation_parameters_2012.json" \Change --dbf_parameters_json to value value from table below then type:
--output_filename="path/to/output_filename.pkl"Change --output_filename to real path on your computer
| Algorithm | Parameters |
|---|---|
| SSD + DeNet, mAP-s(%) | |
| DBF | SSD_DeNet_0.015_single_cross_validation_parameters_2012.json |
| SSD + DeNet, mAP(%) | |
| DBF | SSD_DeNet_0.015_multiple_cross_validation_parameters_2012.json |
| SSD + DeNet + Faster R-CNN, mAP-s(%) | |
| DBF | SSD_DeNet_Faster_R-CNN_0.015_single_cross_validation_parameters_2012.json |
| SSD + DeNet + Faster R-CNN, mAP(%) | |
| DBF | SSD_DeNet_Faster_R-CNN_0.015_multiple_cross_validation_parameters_2012.json |
- Convert "path/to/output_filename.pkl" to PASCAL VOC 2012 submission format by running:
python ./detections_to_PASCAL_VOC_2012_submission.py \
--detections_filename="detections_path.pkl" \Change --detections_filename value from table below then type:
--submission_folder="path/to/submission_folder"Change --submission_folder value to real path on your computer
-
Archive "path/to/submission_folder/results" as .tar.gz
-
Upload "path/to/submission_folder/results.tar.gz" to PASCAL VOC 2012 evaluation server
- Get NMS detections running the following command:
python ./validate_NMS.py \
--imagenames_filename="./PASCAL_VOC_files/imagenames_2012_test.txt" \
--pickled_annots_filename="./PASCAL_VOC_files/annots_2012_test.pkl" \
--dataset_dir="path/to/VOC2012 test/VOC2012" \Change --dataset_dir value to real path to /VOC2012 test/VOC2012 then type:
--nms_parameters_json="./Algorithm_parameters/NMS/SSD_DeNet_0.015_single_cross_validation_parameters_2012.json" \Change --nms_parameters_json to value value from table below then type:
--output_filename="path/to/output_filename.pkl"Change --output_filename to real path on your computer
To get different detectors combinations results in NMS use parameters from the table:
| Algorithm | Parameters |
|---|---|
| SSD + DeNet, mAP-s(%) | |
| NMS | SSD_DeNet_0.015_single_cross_validation_parameters_2012.json |
| SSD + DeNet, mAP(%) | |
| NMS | SSD_DeNet_0.015_multiple_cross_validation_parameters_2012.json |
| SSD + DeNet + Faster R-CNN, mAP-s(%) | |
| NMS | SSD_DeNet_Faster_R-CNN_0.015_single_cross_validation_parameters_2012.json |
| SSD + DeNet + Faster R-CNN, mAP(%) | |
| NMS | SSD_DeNet_Faster_R-CNN_0.015_multiple_cross_validation_parameters_2012.json |
- Convert "path/to/output_filename.pkl" to PASCAL VOC 2012 submission format by running:
python ./detections_to_PASCAL_VOC_2012_submission.py \
--detections_filename="detections_path.pkl" \Change --detections_filename value from table below then type:
--submission_folder="path/to/submission_folder"Change --submission_folder value to real path on your computer
-
Archive "path/to/submission_folder/results" as .tar.gz
-
Upload "path/to/submission_folder/results.tar.gz" to PASCAL VOC 2012 evaluation server