ros2/rmw_cyclonedds

Odd behavior with multiple interfaces

mikeferguson opened this issue · 4 comments

mikeferguson commented

Bug report

I'm seeing a very strange issue where the packets from my computer are using the wrong IP for the interface.

Required Info:

  • Operating System:
    • Ubuntu 22.04
  • Installation type:
    • Binaries
  • Version or commit hash:
    • Humble
  • DDS implementation:
    • CycloneDDS, 0.9.1

Steps to reproduce issue

My desktop has two interfaces:

  • wlan0 has IP address of 10.10.0.106 - netmask 255.255.255.0
  • eth1 has interface of 10.0.0.7 - netmask 255.255.255.0

My robot is at 10.10.0.245.

If I run "ddsperf sanity" - the desktop and robot can communicate without issue.

If I instead run anything ROS2 on the robot (doesn't matter if it is my drivers or the demo/talker), I can't communicate with it from the desktop.

Opening wireshark, I found that the SPDP packets being sent over wlan0 are coming from 10.0.0.7 - that's the IP address of the other interface. If I disable eth1, things work, but otherwise, I've found no way to fix the IP that the packets are coming from.

Config on the desktop:

CYCLONEDDS_URI='<CycloneDDS><Domain><General><Interfaces><NetworkInterface name="wlan0" priority="default"/></Interfaces></General></Domain></CycloneDDS>'

Expected behavior

Packets sent over wlan0 would use the correct IP for the interface

Actual behavior

The packets have an IP address that is not reachable from outside the machine (but only in ROS2, not in "ddsperf sanity")

@eboasson I opened this here, because I think the issue is likely in the RMW since "ddsperf sanity" works - but maybe I'm missing something.

eboasson commented

I too would guess it is caused by the RMW layer, but in Humble it doesn't play that many dangerous games with the configuration, it really looks like it only fiddles with it if ROS2_LOCALHOST is set. Rolling is much more scary in that respect.

Can you share a trace of Cyclone? I'm especially the configuration and network choices at the beginning because those choices should determine all the addresses it eventually uses.

mikeferguson commented

Do you have a pointer to docs on how to get that trace?

eboasson commented

I do somewhere, but it is easier to edit your CYCLONEDDS_URI bit:

CYCLONEDDS_URI='<CycloneDDS><Domain><General><Interfaces><NetworkInterface name="wlan0" priority="default"/></Interfaces></General><Tracing><Category>trace</Category><OutputFile>cdds.log.${CYCLONEDDS_PID}</OutputFile</Tracing></Domain></CycloneDDS>'

(I hope I didn't make any typos ...)

By the way, you're allowed to abbreviate element and attribute names, reduce the closing tags to </> and skip the CycloneDDS and Domain elements when you put the configuration in the environment variable, because that's so convenient 😃 (You have to do it properly when you put it in an XML file, we can't be too lenient.)

mikeferguson commented

I finally went back to make the trace - but I can no longer reproduce the issue. Will re-open if it re-occurs and I get a trace.