NVIDIA-ISAAC-ROS/isaac_ros_dnn_inference

Subscribe to multiple topics?

jrb-cobot opened this issue · 1 comments

jrb-cobot commented

Hello,

I hope you are doing well. I am currently working with a TensorRT node in a ROS2 environment and have encountered a situation where I would like to receive input from multiple camera topics. I wanted to check if it is possible to have the same TensorRT node subscribe to multiple topics and, if so, how I can identify which output corresponds to the input topic.

Current Setup:

  • I have one TensorRT engine plan.
  • I have a TensorRT node that publishes output data to a topic named outputs.
  • There are two camera topics available: /cam1/color/image_raw and /cam2/color/image_raw.
  • I have a DNN image encoder node that encodes these as tensors. Let's say it's /cam1/tensor and /cam2/tensor

Objective:

  • To have the TensorRT node subscribe to both /cam1/tensor and /cam2/tensor`.
  • To determine which input topic (/cam1/color/image_raw or /cam2/color/image_raw) the published output in the outputs topic corresponds to.
    I would be grateful if you could provide guidance on how to achieve this. If this is not feasible, I would appreciate any recommendations for alternative approaches.

Thank you for your time and assistance.

gordongrigor commented

DNN Stereo Disparity does exactly this, as the ESS DNN requires a left & right camera frame for inference. See the section on topics subscribed as it takes two camera frames as inputs.

You did not mention if the DNN you are using requires these images from camera 0 & 1 for inference one or both images, or if you are attempting to batch inference for higher performance, however for either case you need to group the needed input to the DNN image encoder you have before passing tensors to TensorRT.

Noting that DNN inference performance improves with batching, hence if you have a forward and rear camera that needs the same inference operation, it's more efficient to batch both camera's into TensorRT, than a single camera at a time. This would also apply to a camera belt surrounding a robot of 4 cameras, where inference could be batched for all 4 frames at once.