This ROS2 node uses the NVIDIA GPU-accelerated AprilTags library to detect AprilTags in images and publishes their poses, IDs, and additional metadata. This has been tested on ROS2 (Foxy) and should build and run on x86_64 and aarch64 (Jetson). It is modeled after and comparable to the ROS2 node for CPU AprilTags detection.
For more information on the Isaac GEM that this node is based off of, see the latest Isaac SDK documentation here.
For more information on AprilTags themselves, including the paper and the reference CPU implementation, click here.
This Isaac ROS package is designed and tested to be compatible with ROS2 Foxy on Jetson hardware.
- AGX Xavier or Xavier NX
- JetPack 4.6
- CUDA 10.2+ supported discrete GPU
- VPI 1.1.11
- Ubuntu 20.04+
Note: For best performance on Jetson, ensure that power settings are configured appropriately (Power Management for Jetson).
Precompiled ROS2 Foxy packages are not available for JetPack 4.6 (based on Ubuntu 18.04 Bionic). You can either manually compile ROS2 Foxy and required dependent packages from source or use the Isaac ROS development Docker image from Isaac ROS Common.
You must first install the Nvidia Container Toolkit to make use of the Docker container development/runtime environment.
Configure nvidia-container-runtime
as the default runtime for Docker by editing /etc/docker/daemon.json
to include the following:
"runtimes": {
"nvidia": {
"path": "nvidia-container-runtime",
"runtimeArgs": []
}
},
"default-runtime": "nvidia"
and then restarting Docker: sudo systemctl daemon-reload && sudo systemctl restart docker
Run the following script in isaac_ros_common
to build the image and launch the container:
$ scripts/run_dev.sh <optional path>
You can either provide an optional path to mirror in your host ROS workspace with Isaac ROS packages, which will be made available in the container as /workspaces/isaac_ros-dev
, or you can setup a new workspace in the container.
Note: isaac_ros_common
is used for running tests and/or creating a development container. It also contains VPI Debian packages that can be installed natively on a development machine without the container.
- Create a ROS2 workspace if one is not already prepared:
mkdir -p your_ws/src
Note: The workspace can have any name; the quickstart assumes you name ityour_ws
. - Clone this package repository to
your_ws/src/isaac_ros_apriltag
. Check that you have Git LFS installed before cloning to pull down all large files.
sudo apt-get install git-lfs
cd your_ws/src && git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_apriltag
- Build the workspace:
cd your_ws && colcon build --symlink-install
- Source the workspace (in a new terminal):
cd your_ws && source install/setup.bash
- (Optional) Run tests to verify complete and correct installation:
colcon test
- Start
isaac_ros_apriltag
using the prebuilt executable:
ros2 run isaac_ros_apriltag isaac_ros_apriltag
- In a separate terminal, spin up a calibrated camera publisher to
/image_rect
and/camera_info
using any package (for example,v4l2_camera
):
ros2 run v4l2_camera v4l2_camera_node --ros-args -r /image_raw:=/image_rect
- Observe the AprilTag detection output
/tag_detections
on a separate terminal with the command:
ros2 topic echo /tag_detections
You will need to calibrate the intrinsics of your camera if you want the node to determine 3D poses for tags instead of just detection and corners as 2D pixel coordinates. See here for more details.
- Add a dependency on
isaac_ros_apriltag
toyour_package/package.xml
andyour_package/CMakeLists.txt
. The originalapriltag_ros
dependency may be removed entirely. - Change the package and plugin names in any
*.launch.py
launch files to useisaac_ros_apriltag
andAprilTagNode
, respectively.
isaac_ros_image_pipeline
: Accelerated metapackage offering similar functionality to the standard CPU-basedimage_pipeline
metapackageisaac_ros_common
: Utilities for robust ROS2 testing, in conjunction withlaunch_test
This tutorial will help you quickly run and experiment with the full Isaac ROS Apriltag pipeline, from camera frames to tag detections.
- Complete the Quickstart steps above.
- Connect a compatible camera to your Jetson and set up the camera publisher stream. Your camera vendor may offer a specific ROS2-compatible camera driver package. Alternatively, many generic cameras are compatible with the
v4l2_camera
package.
Important: Ensure that the camera stream publishesImage
andCameraInfo
pairs to the topics/image_raw
and/camera_info
, respectively. - Ensure that your workspace has been built and sourced, if you have not done so already:
cd your_ws && colcon build --symlink-install && source install/setup.bash
- Finally, launch the pre-composed pipeline launchfile:
ros2 launch isaac_ros_apriltag isaac_ros_apriltag_pipeline.launch.py
-
Complete the Quickstart steps above.
-
Ensure that your workspace has been built and sourced, if you have not done so already:
cd your_ws && colcon build --symlink-install && source install/setup.bash
-
Launch the pre-composed pipeline launchfile:
ros2 launch isaac_ros_apriltag isaac_ros_apriltag_isaac_sim_pipeline.launch.py
-
Make sure you have Isaac Sim set up correctly and choose the appropriate working environment[Native/Docker&Cloud]. For this walkthrough, we are using the native workstation setup for Isaac Sim.
-
See Running For The First Time section to launch Isaac Sim from the app launcher and click on the Isaac Sim button.
-
Set up the Isaac Sim ROS2 bridge as described here.
-
Connect to the Nucleus server as shown in the Getting Started section if you have not done it already.
-
Open up the Isaac ROS Common USD scene located at:
omniverse://<your_nucleus_server>/Isaac/Samples/ROS/Scenario/carter_warehouse_apriltags_worker.usd
.And wait for it to load completely.
-
Press Play to start publishing data from Isaac Sim.
- In a separate terminal, run RViz to visualize the apriltag detections:
rviz2
- Add the tf tree in the Displays RViz panel.
- Set the Fixed frame in the Global Options to chassis_link.
- You should see the pose of the tags in RVIZ:
- If you prefer to observe the Apriltag output in a text mode, on a separate terminal, echo the contents of the
/tag_detections
topic with the following command:
ros2 topic echo /tag_detections
The following instructions are for a setup where we can run the sample on a Jetson device and Isaac Sim on an x86 machine. We will use the ROS_DOMAIN_ID environment variable to have a separate logical network for Isaac Sim and the sample application.
NOTE: Before executing any of the ROS commands, make sure to set the ROS_DOMAIN_ID variable first.
-
Complete step 4 of Tutorial with Isaac Sim section if you have not done it already.
-
Open the location of the Isaac Sim package in the terminal by clicking the Open in Terminal button.
-
In the terminal opened by the previous step, set the ROS_DOMAIN_ID as shown:
export ROS_DOMAIN_ID=<some_number>
-
Launch Isaac Sim from the script as shown:
./isaac-sim.sh
-
Continue with step 6 of Tutorial with Isaac Sim section. Make sure to set the ROS_DOMAIN_ID variable before running the sample application.
Now that you have successfully launched the full Isaac ROS Apriltag pipeline, you can easily adapt the provided launchfile to integrate with your existing ROS2 environment.
Alternatively, since the AprilTagNode
is provided as a ROS2 Component, you can also compose the accelerated Apriltag processor directly into an existing executable.
The isaac_ros_apriltag
package offers functionality for detecting poses from AprilTags in the frame. It largely replaces the apriltag_ros
package, though an included dependency on the ImageFormatConverterNode
plugin of the isaac_ros_image_proc
package also functions as a way to replace the CPU-based image format conversion in cv_bridge
.
Component | Topics Subscribed | Topics Published | Parameters |
---|---|---|---|
AprilTagNode |
camera/image_rect , camera/camera_info : The input camera stream |
tag_detections : The detection message array tf : The tag poses |
family : The tag family for the detector (this value can only be 36h11 at this time) size : The tag edge size in meters, assuming square markers max_tags : The maximum number of tags to be detected, which is 20 by default |
Date | Changes |
---|---|
2021-11-15 | Isaac Sim HIL documentation update |
2021-11-15 | Added launch file to work with Isaac Sim |
2021-10-20 | Migrated to NVIDIA-ISAAC-ROS |
2021-08-11 | Initial release to NVIDIA-AI-IOT |