Reduced point cloud frequency with ROS2 driver vs plain C++ client
Closed this issue · 8 comments
Describe the bug
ROS2 driver does not achieve desired frequency. In lidar mode 1024x20 for instance, I am getting about 15hz, the proportion is similar in other modes (e.g. 7 hz for 1024x10). Setting rx buffer size and kernel buffer sizes appears to have no effect on this.
Neither does changing the point type (e.g. xyz).
Sometimes I see A message was lost!!! in the output of ros2 topic echo.
Meanwhile a custom C++ client based on the ouster-sdk on the same laptop and the same ethernet adapter achieves the desired frame rate with no problem, so it's not a networking issue.
What steps can I take to improve this? Is there documentation?
Platform (please complete the following information):
- Ouster Sensor? OS1-128 rev7
- Ouster Firmware Version? v3.0.1
- ROS version/distro? humble
- Operating System? ubuntu mate 22.04
- Machine Architecture? x64
- git commit hash (if not the latest): 5dd8555
I can answer this myself more or less. I read somewhere the suggestion to use the Cyclone middleware, so I ran
sudo apt -y install ros-humble-rmw-cyclonedds-cpp
Then made sure to quit my shell session, and run export RMW_IMPLEMENTATION=rmw_cyclonedds_cpp before opening a new tmux session and sourcing the ROS environment. That made the frequency shoot up to 20hz immediately.
Could some troubleshooting FAQ be added on this?
Yeah generally I recommend using CycloneDDS against the ROS2 driver. I want to add a note to our main README.md since this comes very often. At the same time I still have some optimization that I want to implement to improve the performance for FastDDS.
I retested right now with the cyclone middleware, and I'm still not getting full frequency, I'm guessing I made other changes to speed it up. That, or my firmware upgrade changed the device's behaviour.
Is there any recommendation on kernel receive buffer sizes or other settings one should apply on a connected system?
Can confirm. I made these two changes which bumped the frequency up. Not sure which of these was needed:
# kernel receive buffer size
sudo sysctl -w net.core.rmem_max=16777216
sudo sysctl -w net.core.rmem_default=16777216
# hardware ring buffer size (?)
sudo ethtool -G enx00e04ca0ba32 rx 2048
yeah that's the recommendation per https://community.ouster.com/t/increase-receive-buffer-to-eliminate-packet-drop-problem-on-some-windows-computers/84/3 besides using CycloneDDS for ROS2
totally, I actually want to include a bash script as part of the driver that users can invoke to configure the system properly for them, since this is rather a recurring issue.