ros-visualization/rviz

Crash due to interference between MarkerArrayDisplay and PlanningSceneDisplay

simonschmeisser opened this issue · 4 comments

simonschmeisser commented

We recently started seeing frequent crashes in our simulation frontend. It happens around every second pick cycle. The same or similar crashes happen when I add the same Displays in standalone rviz.

We have one PlanningSceneDisplay and one MarkerArrayDisplay with the marker displaying the ground truth of objects that are then rendered (externally) to stereo images, recognized and added to the PlanningScene. When our simulated robot picks one object the simulation gets a message and publishes a new MarkerArray excluding the picked object. After receiving this MarkerArray message it crashes in the next render cycle.

All stack traces seem to reference Ogre::Materials, could there be a name clash? We have not seen this crash with ogre 1.9

sometimes it prints

corrupted size vs. prev_size
Abgebrochen (Speicherabzug geschrieben)

and sometimes only

Speicherzugriffsfehler (Speicherabzug geschrieben)

The following are backtraces from rviz

https://gist.github.com/simonschmeisser/e3982129b5e7a6fed69518addb7d11ac
https://gist.github.com/simonschmeisser/07f1026b01250c6574756d7225766f5b
https://gist.github.com/simonschmeisser/a8f0888eb9b7be3abc57dfb6ce667c12
https://gist.github.com/simonschmeisser/17eab5f0c6a48c163f9fc4d36ee5fedf

sample MarkerArray message:

https://gist.github.com/simonschmeisser/ba52a21844d19ad10112252c98c3d60b

Your environment

  • OS Version: Ubuntu 22.04
  • ROS Distro: one
  • RViz, Qt, OGRE, OpenGl version as printed by rviz:

[INFO] [1686902695.832477086]: rviz version 1.14.20
[INFO] [1686902695.832510709]: compiled against Qt version 5.15.7
[INFO] [1686902695.832516630]: compiled against OGRE version 1.12.13 (Rhagorthua)
[INFO] [1686902695.842539504]: Forcing OpenGl version 0.
[INFO] [1686902696.082908729]: Stereo is NOT SUPPORTED
[INFO] [1686902696.082960315]: OpenGL device: RENOIR (renoir, LLVM 15.0.7, DRM 3.47, 5.19.0-43-generic)
[INFO] [1686902696.082974171]: OpenGl version: 4,6 (GLSL 4,6) limited to GLSL 1.4 on Mesa system.
```

simonschmeisser commented

I cannot reproduce any longer after recompiling MoveIt and Rviz locally with asan enabled. Maybe I messed up something when creating our packages, will rebuild everything

simonschmeisser commented

I rebuild most things but these crashes persist when using our packages. When building locally with asan everything works fine. I'm somewhat lost. Any ideas @rhaschke ? I suspect that "something" calls the wrong destructor of "something" ogre but I'm lost as too what that might be.

Ogre is 1.12.13

I build in a docker container containing ros-one-* packages from Michael and in there create an overlay workspace containing:

fcl 
geometric_shapes
industrial_robot_status_controller
kuka_test_resources
moveit_msgs
moveit_resources
octomap_rviz_plugins
panda_moveit_config
rviz_visual_tools
trac_ik
ensenso-profiler
gazebo_ros_pkgs
industrial_core
interactive_markers
moveit
moveit_opw_kinematics_plugin
moveit_visual_tools
ompl
rviz
srdfdom

This workspace then gets installed as deb package to /opt/isys-vision/mikado and all of our custom software packages are build on another docker container with this package installed.

simonschmeisser commented

I tried the same procedure with Ogre 13.6.4 and Ogre 1.9 (from debian). With Ogre 13 I got similar crashes to the above ones. With Ogre 1.9 everything seems to work (except for known NVidia bugs ...)

rhaschke commented

Sorry for the late reply on this: Is it possible that you mix up ogre-dev 1.9 and 1.13? Michael's ros-one-* surely pulls in ogre-dev 1.9 via gazebo_ros_pkgs and gazebo_dev...