Function Overview

BEV perception algorithm is a BEV multi-task model trained on the nuscenes dataset using Horizon OpenExplorer.

The algorithm takes 6 sets of image data as input, including frontal, front left, front right, rear, rear left, and rear right views. The model outputs 10 categories of objects along with their corresponding 3D detection boxes, including obstacles, various types of vehicles, traffic signs, as well as semantic segmentation of lane lines, sidewalks, and road edges.

This example uses local image data as input, performs algorithm inference using BPU, publishes images of algorithm perception results, and renders them on a PC browser.

Bill of Materials

Usage Instructions

Function Installation

Run the following commands in the terminal of the RDK system for quick installation:

tros foxy:

sudo apt update
sudo apt install -y tros-hobot-bev
sudo apt install -y tros-websocket

tros humble:

sudo apt update
sudo apt install -y tros-humble-hobot-bev
sudo apt install -y tros-humble-websocket

Prepare for Data Backfilling

Run the following commands in the terminal of the RDK system to download and unzip the dataset:

# Board-side dataset download
wget http://sunrise.horizon.cc/TogetheROS/data/hobot_bev_data.tar.gz

# Unzip
mkdir -p hobot_bev_data
tar -zxvf hobot_bev_data.tar.gz -C hobot_bev_data

# After extraction, the dataset will be available in the hobot_bev_data/data path

Launch Algorithm and Image Visualization

Run the following commands in the terminal of the RDK system to start the algorithm and visualization:

tros foxy:

# Configure the tros.b environment
source /opt/tros/setup.bash

# Start the websocket service
ros2 launch websocket websocket_service.launch.py

# Start the execution script and specify the dataset path
ros2 launch hobot_bev hobot_bev.launch.py image_pre_path:=hobot_bev_data/data

tros humble:

# Configure the tros.b humble environment
source /opt/tros/humble/setup.bash

# Start the websocket service
ros2 launch websocket websocket_service.launch.py

# Start the execution script and specify the dataset path
ros2 launch hobot_bev hobot_bev.launch.py image_pre_path:=hobot_bev_data/data

After successful launch, open a browser on the same network computer and visit the IP address of RDK http://IP:8000 (where IP is the IP address of RDK) to see the real-time visualization of the algorithm:

Interface Description

Topics

Name	Message Type	Description
/image_jpeg	sensor_msgs/msg/Image	Periodically publishes image topic in jpeg format

Parameters

Name	Parameter Value	Description
image_pre_path	Path to the actual location of the playback dataset	Path to the playback dataset

HorizonRDK/hobot_bev