/perception_eval_rs

Primary LanguageRustMIT LicenseMIT

perception_eval_rs

Rust binding of tier4/autoware_perception_evaluation. perception_eval_rs provides modules to evaluate perception outputs recognized by your object recognition functions.

Support

  • ✔️ : Completed
  • ✅ : WIP / TODO
  • ❌ : Not implemented yet

Dataset

Format Description Support
NuScenes NuScenes format ✅
NuImages NuImages format ❌

Evaluation tasks

Task Metrics Description Support
Detection mAP, mAPH 3D detection ✔️
Tracking CLEAR 3D tracking ❌
Prediction ADE, FDE, MissRate 3D motion prediction ❌
Detection2D mAP 2D detection ❌
Tracking2D CLEAR 2D tracking ❌

Object type

Name Description Support
DynamicObject 3D object ✔️
DynamicObject2D 2D object ❌

Coordinates system

Name Support
BaseLink ✔️
Map ❌
Camera ❌

Evaluation manager & configuration

Name Description Support
PerceptionEvaluationManager Manager to evaluate perception tasks ✔️
PerceptionEvaluationConfig Configuration of perception evaluation ✔️

Matching objects

Name Description Support
CenterDistance Euclidean distance between center of objects ✔️
PlaneDistance RMS score of nearest two planes ✔️
Iou2D 2D IoU score ✔️
Iou3D 3D IoU score ✔️

Filter objects

Parameter Description Support
target_labels List of labels to keep ✔️
max_x_positions List of maximum x positions for each label ✔️
max_y_positions List of maximum y positions for each label ✔️
min_point_numbers List of minimum number of points the object's box must contain ✔️
target_uuids List of instance IDs to keep ✔️

Object results

Name Description Support
PerceptionResult Matching pair of estimated and ground truth objects ✔️
PerceptionFrameResult Set of PerceptionResult ✔️

Metrics score

Name Description Support
MetricsScore Calculate score for each metrics ✔️

Documents

Run the following code to see the document.

$ cargo doc --open

Examples

Run the following code to run examples. See examples.

# if you want see details
# $ export RUST_BACKTRACE=1 [or "full"]
$ cargo run --example <FILE_NAME> [-- <ARGUMENTS>]

Unit tests

Run the following code to execute unit tests written in doc-strings.

# test all modules (public&private)
cargo test -- [OPTIONS]

# test only public modules that tests are written in doc-strings
cargo test --doc --package perception-eval -- [OPTIONS]

References