/isaac_ros_argus_camera

ROS2 packages based on NVIDIA libArgus library for hardware-accelerated CSI camera support.

Primary LanguageC++OtherNOASSERTION

Isaac ROS Argus Camera

Overview

This repository provides the Isaac ROS Argus node, which enables ROS developers to use monocular and stereo cameras connected to Jetson platforms over CSI. The node uses the Libargus API internally for acquiring images and associated metadata from camera devices.

Isaac ROS NITROS Acceleration

This package is powered by NVIDIA Isaac Transport for ROS (NITROS), which leverages type adaptation and negotiation to optimize message formats and dramatically accelerate communication between participating nodes.

Table of Contents

Latest Update

Update 2022-08-31: Update to be compatible with JetPack 5.0.2

Supported Platforms

This package is designed and tested to be compatible with ROS2 Humble running on Jetson with off-the-shelf cameras from NVIDIA partners (see the Reference Cameras section for more details).

Note: x86_64 system is not supported.

Note: Versions of ROS2 earlier than Humble are not supported. This package depends on specific ROS2 implementation features that were only introduced beginning with the Humble release.

Platform Hardware Software Notes
Jetson Jetson Orin
Jetson Xavier		 JetPack 5.0.2 The CSI camera device needs to be connected and presented as a video device (e.g. /dev/video0).

For best performance, ensure that power settings are configured appropriately.

Docker

To simplify development, we strongly recommend leveraging the Isaac ROS Dev Docker images by following these steps. This will streamline your development environment setup with the correct versions of dependencies on Jetson platforms.

Note: All Isaac ROS Quickstarts, tutorials, and examples have been designed with the Isaac ROS Docker images as a prerequisite.

Reference Cameras

NVIDIA has worked with our camera partners to provide the modules listed below which are compatible with the Isaac ROS Argus Camera package.

Camera Type Connector Resolution/FrameRate H/V FOV Shutter Supplier, HW Part, Link to purchase
Stereo Camera/Color GMSL 1920x1200@60fps 121/73 Global LI-AR0234CS-STEREO-GMSL2
Fisheye Camera/Color GMSL 1920x1200@60fps 202/127 Global LI-AR0234CS-GMSL2-OWL
Monocular Camera/Color CSI 4056x3040@60fps 140/103 Global IMX477 -140FOH
Monocular Camera/Color CSI 3264x2464@21fps 90/90 Rolling LI-NANO-CB-IMX219-090H

Quickstart

  1. Set up your development environment by following the instructions here.

  2. Clone this repository and its dependencies under ~/workspaces/isaac_ros-dev/src.

    cd ~/workspaces/isaac_ros-dev/src
    git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common
    git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_nitros
    git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_argus_camera

  3. Launch the Docker container using the run_dev.sh script:

    cd ~/workspaces/isaac_ros-dev/src/isaac_ros_common && \
  ./scripts/run_dev.sh

  4. Inside the container, build and source the workspace:

    cd /workspaces/isaac_ros-dev && \
  colcon build --symlink-install && \
  source install/setup.bash

  5. (Optional) Run tests to verify complete and correct installation:

    colcon test --executor sequential

  6. Run the following launch files to spin up a demo of this package:

    ros2 launch isaac_ros_argus_camera isaac_ros_argus_camera_mono.launch.py

  7. Use image_saver to save the output images:

    ros2 run image_view image_saver --ros-args -r image:=/left/image_raw -p filename_format:="left_image.jpg"

Package Reference

Usage

Launch monocular camera

ros2 launch isaac_ros_argus_camera isaac_ros_argus_camera_mono.launch.py

Launch stereo camera

ros2 launch isaac_ros_argus_camera isaac_ros_argus_camera_stereo.launch.py

ROS Parameters

ROS Parameter Type Default Description
camera_id uint 0 The video device index
E.g. /dev/video0
module_id uint 0 The camera module index in the device tree, If there is more than one but the same camera module connected
mode uint 0 The sensor mode supported by the camera sensor and driver
framerate uint 30 The frame rate of the sensor mode
E.g. 30 or 60 fps
camera_type uint 1 0 for Monocular type camera; 1 for Stereo type camera
output_format uint 1 Output color space format from libargus.
0 for YUV420 to RGB888 ; 1 for YUV444 to RGB888
camera_info_url string N/A, optional The URL of camera info .ini file for monocular camera
left_camera_info_url string N/A, optional The URL of camera info .ini file for stereo left
right_camera_info_url string N/A, optional The URL of camera info .ini file for stereo right

Note: To run the stereo camera, two video devices should present for the left and right sensors, respectively (e.g. /dev/video0 and /dev/video1).

ROS Topics Published

ROS Topic Interface Description
left/image_raw sensor_msgs/Image The left image of a stereo pair.
right/image_raw sensor_msgs/Image The right image of a stereo pair.
left/camerainfo sensor_msgs/CameraInfo The left camera model.
right/camerainfo sensor_msgs/CameraInfo The right camera model.

Launch testing

Monocular camera

launch_test src/isaac_ros_argus_camera/isaac_ros_argus_camera/test/isaac_ros_argus_camera_mono_test.py

Stereo camera

launch_test src/isaac_ros_argus_camera/isaac_ros_argus_camera/test/isaac_ros_argus_camera_stereo_test.py

Output Color Space Format

The Isaac ROS Argus node supports the YUV444 and YUV420 colorspaces from libargus and converts it to the RGB888 colorspace as output.

CameraInfo Message

The Isaac ROS Argus node uses the Argus Ext API to retrieve calibration parameters from the camera through the Linux device driver and convert it into CameraInfo messages. Refer to this page for the data structure of the calibration parameters.

Note: Each camera module should have stored the calibration parameters in internal memory like EEPROM, and the device driver must support the Argus Ext API to extract those parameters. Contact your camera vendor to get the required drivers.

Note: If your camera does not support the Argus Ext API, you can also specify a URL to a camera info .ini file parseable by the ROS CameraInfoManager using the camera_info_url parameter on the Isaac ROS Argus node. This will allow you to provide parameters you may have calibrated using the ROS Camera Calibration package, for example.

Note: When the camera_info_url is provided, the loaded parameters override the CameraInfo from Argus Ext API.

Troubleshooting

Isaac ROS Troubleshooting

For solutions to problems with Isaac ROS, please check here.

Argus fails to create capture session

Symptoms

The Isaac ROS Argus node can fail to create a capture session inside the container after the nvargus daemon has crashed. By default, the nvargus daemon is running in background, but it may crash due to other Argus clients. This will prevent Argus camera nodes from creating capture sessions. You may see messages with errors similar to Failed to create capture session.

Solution

Exit the Docker container and restart the nvargus daemon by running sudo service nvargus-daemon restart

Updates

Date Changes
2022-08-31 Update to be compatible with JetPack 5.0.2
2022-06-30 Support NITROS acceleration
2022-03-18 Support CameraInfo from URL
2021-10-20 Initial release