robotology/gazebo-fmi

Investigate OpenModelica FMU segfaults in model with dampers

Closed this issue · 22 comments

While developing the transmission for the examples, for several Modelica models the relative FMU generated by OpenModelica 1.12 was segfault, both if used through the FMILibrary of through FMPy .
I guess this problem was connected to https://trac.openmodelica.org/OpenModelica/ticket/4135 .

Apparently the problem is with models with dampers:

model_with_dampers

model CompliantTransmission
  Modelica.Blocks.Interfaces.RealInput actuatorInput annotation(
    Placement(visible = true, transformation(origin = {-120, 0}, extent = {{-20, -20}, {20, 20}}, rotation = 0), iconTransformation(origin = {-120, 0}, extent = {{-20, -20}, {20, 20}}, rotation = 0)));
  Modelica.Blocks.Interfaces.RealInput jointPosition annotation(
    Placement(visible = true, transformation(origin = {120, -60}, extent = {{-20, -20}, {20, 20}}, rotation = 180), iconTransformation(origin = {120, -60}, extent = {{-20, -20}, {20, 20}}, rotation = 180)));
  Modelica.Blocks.Interfaces.RealInput jointVelocity annotation(
    Placement(visible = true, transformation(origin = {118, 0}, extent = {{-20, -20}, {20, 20}}, rotation = 180), iconTransformation(origin = {118, 0}, extent = {{-20, -20}, {20, 20}}, rotation = 180)));
  Modelica.Blocks.Interfaces.RealOutput jointTorque annotation(
    Placement(visible = true, transformation(origin = {110, 60}, extent = {{-10, -10}, {10, 10}}, rotation = 0), iconTransformation(origin = {110, 60}, extent = {{-10, -10}, {10, 10}}, rotation = 0)));
  Modelica.Mechanics.Rotational.Components.AngleToTorqueAdaptor angleToTorqueAdaptor1(use_a = false, use_w = true) annotation(
    Placement(visible = true, transformation(origin = {41, 0}, extent = {{-21, -20}, {21, 20}}, rotation = 180)));
  Modelica.Mechanics.Rotational.Sources.Torque torque1 annotation(
    Placement(visible = true, transformation(origin = {-76, 0}, extent = {{-10, -10}, {10, 10}}, rotation = 0)));
  Modelica.Blocks.Math.Gain invertSign(k = -1)  annotation(
    Placement(visible = true, transformation(origin = {70, 16}, extent = {{-10, -10}, {10, 10}}, rotation = 0)));
  Modelica.Mechanics.Rotational.Components.Spring spring1(c = 10)  annotation(
    Placement(visible = true, transformation(origin = {4, 0}, extent = {{-10, -10}, {10, 10}}, rotation = 0)));
  Modelica.Mechanics.Rotational.Components.Inertia inertia annotation(
    Placement(visible = true, transformation(origin = {-46, 0}, extent = {{-10, -10}, {10, 10}}, rotation = 0)));
  Modelica.Mechanics.Rotational.Components.Damper damper1 annotation(
    Placement(visible = true, transformation(origin = {-20, 0}, extent = {{-10, -10}, {10, 10}}, rotation = 0)));
equation
  connect(damper1.flange_a, inertia.flange_b) annotation(
    Line(points = {{-30, 0}, {-36, 0}, {-36, 0}, {-36, 0}}));
  connect(damper1.flange_b, spring1.flange_a) annotation(
    Line(points = {{-10, 0}, {-6, 0}, {-6, 0}, {-6, 0}}));
  connect(torque1.flange, inertia.flange_a) annotation(
    Line(points = {{-66, 0}, {-56, 0}}));
  connect(spring1.flange_b, angleToTorqueAdaptor1.flange) annotation(
    Line(points = {{14, 0}, {36, 0}, {36, 0}, {36, 0}}));
  connect(invertSign.y, jointTorque) annotation(
    Line(points = {{82, 16}, {90, 16}, {90, 60}, {110, 60}, {110, 60}}, color = {0, 0, 127}));
  connect(angleToTorqueAdaptor1.tau, invertSign.u) annotation(
    Line(points = {{48, 16}, {58, 16}, {58, 16}, {58, 16}}, color = {0, 0, 127}));
  connect(actuatorInput, torque1.tau) annotation(
    Line(points = {{-120, 0}, {-88, 0}}, color = {0, 0, 127}));
  connect(jointPosition, angleToTorqueAdaptor1.phi) annotation(
    Line(points = {{120, -60}, {80.5, -60}, {80.5, -16}, {49, -16}}, color = {0, 0, 127}));
  connect(jointVelocity, angleToTorqueAdaptor1.w) annotation(
    Line(points = {{118, 0}, {79.5, 0}, {79.5, -6}, {49, -6}}, color = {0, 0, 127}));
  annotation(
    uses(Modelica(version = "3.2.2")));
end CompliantTransmission;

@triccyx If you get some similar problem, check this issue.

I have a crash. But I suspect it is not the same. I don't use dumpers.
My gdb stack:


#0  0x00007fc8b80cbb0b in pool_expand () from /tmp/binaries/linux64/Gazebo.so
#1  0x00007fc8b80cb858 in pool_malloc () from /tmp/binaries/linux64/Gazebo.so
#2  0x00007fc8b80cb983 in real_alloc () from /tmp/binaries/linux64/Gazebo.so
#3  0x00007fc8b80dd317 in simple_alloc_1d_real_array () from /tmp/binaries/linux64/Gazebo.so
#4  0x00007fc8b80df3c5 in array_alloc_scalar_real_array () from /tmp/binaries/linux64/Gazebo.so
#5  0x00007fc8b80c2daa in Gazebo_eqFunction_10 () from /tmp/binaries/linux64/Gazebo.so
#6  0x00007fc8b80c3722 in Gazebo_functionInitialEquations_0 () from /tmp/binaries/linux64/Gazebo.so
#7  0x00007fc8b80c387c in Gazebo_functionInitialEquations () from /tmp/binaries/linux64/Gazebo.so
#8  0x00007fc8b80f85c0 in symbolic_initialization () from /tmp/binaries/linux64/Gazebo.so
#9  0x00007fc8b80f8350 in initialization () from /tmp/binaries/linux64/Gazebo.so
#10 0x00007fc8b80ba0c6 in fmi2EnterInitializationMode () from /tmp/binaries/linux64/Gazebo.so
#11 0x00007fc8b85d3f84 in gazebo_fmi::FMUCoSimulationPrivate::createInstance(double) () from /usr/local/lib/libFMIActuatorPlugin.so
#12 0x00007fc8b85d2b44 in gazebo_fmi::FMUCoSimulation::load(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, double) () from /usr/local/lib/libFMIActuatorPlugin.so
#13 0x00007fc8b85c702c in gazebo_fmi::FMIActuatorPlugin::LoadFMUs(boost::shared_ptr<gazebo::physics::Model>) () from /usr/local/lib/libFMIActuatorPlugin.so
#14 0x00007fc8b85c74b1 in gazebo_fmi::FMIActuatorPlugin::Load(boost::shared_ptr<gazebo::physics::Model>, std::shared_ptr<sdf::Element>) ()
   from /usr/local/lib/libFMIActuatorPlugin.so
#15 0x00007fc99bc2f13c in gazebo::physics::Model::LoadPlugin(std::shared_ptr<sdf::Element>) () from /usr/lib/x86_64-linux-gnu/libgazebo_physics.so.8
#16 0x00007fc99bc332a0 in gazebo::physics::Model::LoadPlugins() () from /usr/lib/x86_64-linux-gnu/libgazebo_physics.so.8
#17 0x00007fc99bc7ce65 in gazebo::physics::World::LoadPlugins() () from /usr/lib/x86_64-linux-gnu/libgazebo_physics.so.8
#18 0x00007fc99bc90d1d in gazebo::physics::World::Step() () from /usr/lib/x86_64-linux-gnu/libgazebo_physics.so.8
#19 0x00007fc99bc9111d in gazebo::physics::World::RunLoop() () from /usr/lib/x86_64-linux-gnu/libgazebo_physics.so.8
#20 0x00007fc99cb55c80 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#21 0x00007fc99bfb66ba in start_thread (arg=0x7fc8d7fff700) at pthread_create.c:333
#22 0x00007fc99c5c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
(gdb)

The stack doesn't show anything after the last gazebo::physics::World::RunLoop().
Perhaps the fmu is not correctly loaded or compiled.
First of all, I'll try without the gearbox.

Which version of OpenModelica are you using? In my case, it was easy to reproduce the crash also using the fmpy simulate <name>.fmu command, from FMPy.

I'm using OMEdit 1.12.0.
At the moment I simply use:
gzserver --verbose -e ode --profile ode_default -slibgazebo_yarp_clock.so isaac-next-proto-modelica-yarp.world
and it crashes with core dump.

Can you verify if it also fails with FMPy ? In that way we can identify that the problem is in OpenModelica. Which option did you use for converting the Model in FMU? In the tests I used the options
buildModelFMU(${_OCM_MODEL_NAME}, fmuType = \"cs\", platforms = {\"static\"}); (see https://github.com/robotology/gazebo-fmi/blob/master/cmake/FMIUtils.cmake#L65) .

For my problem, I suspect I have problems locating the fmu files from
gzserver command line. I'm investigating

You should add the directory containing the FMUs in GAZEBO_RESOURCE_PATH .

Problems in using FMPy:

python -m fmpy.gui
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/usr/local/lib/python2.7/dist-packages/fmpy/gui/main.py", line 16, in
from fmpy.gui.generated.MainWindow import Ui_MainWindow
ImportError: No module named MainWindow

However
fmpy simulate gazebo.fmu
Segmentation fault (core dumped)

In this case gdb can't help as the command fmpy is not an exe
"/usr/local/bin/fmpy": not in executable format: File format not recognized

Yes, fmpy is a Python script. However, this seems a OpenModelica problem. Can you try if changing the parameters of buildModelFMU improve the situation? Otherwise we can also try with the latest dev version in https://openmodelica.org/developersresources/nightly-builds , apparently https://trac.openmodelica.org/OpenModelica/ticket/4135 could be solved in the latest nightly.

Ok but before I want to try with a very simple fmu

As B plan ... for next week
Do you know if exist others tools that do Modelica->fmu? In this case, we can just C&P the Modelica code.

I guess JModelica.org, but I never tried it.

buildModelFMU(Gazebo,"2","cs","",{"static","x86_64-linux-gnu"},true)
fails as usual

but with dynamic linkage is much better:

buildModelFMU(Gazebo,"2","cs","<default>",{"dynamic"},true)
stdout            | info    | The initialization finished successfully without homotopy method.
assert            | debug   | division leads to inf or nan at time 0.174, (a=-3.98694e+303) / (b=1.62e-05), where divisor b is: motor.inductor.L
Segmentation fault (core dumped)

at least do something.
and fmpy

fmpy simulate ffgazebo.fmu 
stdout            | info    | The initialization finished successfully without homotopy method.

Now I will remove the gear that usually gives problems

Unexpected behaviour.
1)I have connected the NullTransmision.fmu to AK_R and control in position to 10deg the AK_R doesn't move that's ok for me As the actuatorInput is not connected.
2)I have modified the NullTransmision.fmu with a const torque of 1N on exit
I have connected the NullTransmision_modified.fmu to AK_R and control in position to 33deg->0deg->-33deg the AK_R. It flickers and also moves apparently to complete partially the test. I expected the AK_R to move up to the joint limit and stop there As the actuatorInput is not connected.

Going back to the fmu with motor we have a problem (similar to the dumper?)

assert | debug | division leads to inf or nan at time 0.174, (a=-5.17538e+303) / (b=1.62e-05), where divisor b is: motor.inductor.L

and then crash. Perhaps the problem is due to wrong inputs to the motor. We can try to change the gain in PID

@triccyx I replied on that specific model on Redmine.

I had similar problem with a simple FMU that I prepared to test #17 . The Modelica model is the following:

stifftransmission

model StiffTransmission
  Modelica.Blocks.Interfaces.RealInput actuatorInput annotation(
    Placement(visible = true, transformation(origin = {-120, 0}, extent = {{-20, -20}, {20, 20}}, rotation = 0), iconTransformation(origin = {-120, 0}, extent = {{-20, -20}, {20, 20}}, rotation = 0)));
  Modelica.Blocks.Interfaces.RealInput jointPosition annotation(
    Placement(visible = true, transformation(origin = {120, -60}, extent = {{-20, -20}, {20, 20}}, rotation = 180), iconTransformation(origin = {120, -60}, extent = {{-20, -20}, {20, 20}}, rotation = 180)));
  Modelica.Blocks.Interfaces.RealInput jointVelocity annotation(
    Placement(visible = true, transformation(origin = {120, -20}, extent = {{-20, -20}, {20, 20}}, rotation = 180), iconTransformation(origin = {120, -20}, extent = {{-20, -20}, {20, 20}}, rotation = 180)));
  Modelica.Blocks.Interfaces.RealOutput jointTorque annotation(
    Placement(visible = true, transformation(origin = {110, 60}, extent = {{-10, -10}, {10, 10}}, rotation = 0), iconTransformation(origin = {110, 60}, extent = {{-10, -10}, {10, 10}}, rotation = 0)));
  Modelica.Mechanics.Rotational.Components.AngleToTorqueAdaptor angleToTorqueAdaptor1(use_a = true, use_w = true) annotation(
    Placement(visible = true, transformation(origin = {41, 0}, extent = {{-21, -20}, {21, 20}}, rotation = 180)));
  Modelica.Mechanics.Rotational.Sources.Torque torque1 annotation(
    Placement(visible = true, transformation(origin = {-76, 0}, extent = {{-10, -10}, {10, 10}}, rotation = 0)));
  Modelica.Blocks.Math.Gain invertSign(k = -1) annotation(
    Placement(visible = true, transformation(origin = {70, 60}, extent = {{-10, -10}, {10, 10}}, rotation = 0)));
  Modelica.Mechanics.Rotational.Components.Inertia inertia(J = 0.01) annotation(
    Placement(visible = true, transformation(origin = {-10, 0}, extent = {{-10, -10}, {10, 10}}, rotation = 0)));
  Modelica.Blocks.Interfaces.RealInput jointAcceleration annotation(
    Placement(visible = true, transformation(origin = {120, 20}, extent = {{-20, -20}, {20, 20}}, rotation = 180), iconTransformation(origin = {120, 20}, extent = {{-20, -20}, {20, 20}}, rotation = 180)));
equation
  connect(inertia.flange_b, angleToTorqueAdaptor1.flange) annotation(
    Line(points = {{0, 0}, {36, 0}}));
  connect(torque1.flange, inertia.flange_a) annotation(
    Line(points = {{-66, 0}, {-20, 0}}));
  connect(jointAcceleration, angleToTorqueAdaptor1.a) annotation(
    Line(points = {{120, 20}, {80, 20}, {80, 6}, {50, 6}, {50, 6}}, color = {0, 0, 127}));
  connect(angleToTorqueAdaptor1.tau, invertSign.u) annotation(
    Line(points = {{48, 16}, {53, 16}, {53, 60}, {58, 60}}, color = {0, 0, 127}));
  connect(invertSign.y, jointTorque) annotation(
    Line(points = {{81, 60}, {110, 60}}, color = {0, 0, 127}));
  connect(jointVelocity, angleToTorqueAdaptor1.w) annotation(
    Line(points = {{120, -20}, {83.5, -20}, {83.5, -6}, {49, -6}}, color = {0, 0, 127}));
  connect(actuatorInput, torque1.tau) annotation(
    Line(points = {{-120, 0}, {-88, 0}}, color = {0, 0, 127}));
  connect(jointPosition, angleToTorqueAdaptor1.phi) annotation(
    Line(points = {{120, -60}, {80.5, -60}, {80.5, -16}, {49, -16}}, color = {0, 0, 127}));
  annotation(
    uses(Modelica(version = "3.2.2")));
end StiffTransmission;

With OpenModelica 1.12, the FMU was segfaulting with this backtrace (obtained with valgrind):

[Dbg] [ServerFixture.cc:209] ServerFixture load in 10.4 seconds, timeout after 600 seconds
==20958== Thread 44:
==20958== Invalid read of size 8
==20958==    at 0x5AE62F2B: pool_expand (in /tmp/binaries/linux64/StiffTransmission.so)
==20958==    by 0x5AE62C77: pool_malloc (in /tmp/binaries/linux64/StiffTransmission.so)
==20958==    by 0x5AE62DA2: real_alloc (in /tmp/binaries/linux64/StiffTransmission.so)
==20958==    by 0x5AE74756: simple_alloc_1d_real_array (in /tmp/binaries/linux64/StiffTransmission.so)
==20958==    by 0x5AE76824: array_alloc_scalar_real_array (in /tmp/binaries/linux64/StiffTransmission.so)
==20958==    by 0x5AE5F620: StiffTransmission_eqFunction_3 (in /tmp/binaries/linux64/StiffTransmission.so)
==20958==    by 0x5AE5F876: StiffTransmission_functionInitialEquations_0 (in /tmp/binaries/linux64/StiffTransmission.so)
==20958==    by 0x5AE5F8FB: StiffTransmission_functionInitialEquations (in /tmp/binaries/linux64/StiffTransmission.so)
==20958==    by 0x5AE8FA4F: symbolic_initialization (in /tmp/binaries/linux64/StiffTransmission.so)
==20958==    by 0x5AE8F7DF: initialization (in /tmp/binaries/linux64/StiffTransmission.so)
==20958==    by 0x5AE59765: fmi2EnterInitializationMode (in /tmp/binaries/linux64/StiffTransmission.so)
==20958==    by 0x5C9B049D: gazebo_fmi::FMUCoSimulationPrivate::createInstance(double) (FMUCoSimulation.cc:79)
==20958==  Address 0x10 is not stack'd, malloc'd or (recently) free'd
==20958== 
==20958== 
==20958== Process terminating with default action of signal 11 (SIGSEGV)
==20958==  Access not within mapped region at address 0x10
==20958==    at 0x5AE62F2B: pool_expand (in /tmp/binaries/linux64/StiffTransmission.so)
==20958==    by 0x5AE62C77: pool_malloc (in /tmp/binaries/linux64/StiffTransmission.so)
==20958==    by 0x5AE62DA2: real_alloc (in /tmp/binaries/linux64/StiffTransmission.so)
==20958==    by 0x5AE74756: simple_alloc_1d_real_array (in /tmp/binaries/linux64/StiffTransmission.so)
==20958==    by 0x5AE76824: array_alloc_scalar_real_array (in /tmp/binaries/linux64/StiffTransmission.so)
==20958==    by 0x5AE5F620: StiffTransmission_eqFunction_3 (in /tmp/binaries/linux64/StiffTransmission.so)
==20958==    by 0x5AE5F876: StiffTransmission_functionInitialEquations_0 (in /tmp/binaries/linux64/StiffTransmission.so)
==20958==    by 0x5AE5F8FB: StiffTransmission_functionInitialEquations (in /tmp/binaries/linux64/StiffTransmission.so)
==20958==    by 0x5AE8FA4F: symbolic_initialization (in /tmp/binaries/linux64/StiffTransmission.so)
==20958==    by 0x5AE8F7DF: initialization (in /tmp/binaries/linux64/StiffTransmission.so)
==20958==    by 0x5AE59765: fmi2EnterInitializationMode (in /tmp/binaries/linux64/StiffTransmission.so)
==20958==    by 0x5C9B049D: gazebo_fmi::FMUCoSimulationPrivate::createInstance(double) (FMUCoSimulation.cc:79)
==20958==  If you believe this happened as a result of a stack
==20958==  overflow in your program's main thread (unlikely but
==20958==  possible), you can try to increase the size of the
==20958==  main thread stack using the --main-stacksize= flag.
==20958==  The main thread stack size used in this run was 8388608.

With the nightly OpenModelica 1.13.0~dev-1195-g6d891ea , everything works fine, so I guess that the fixes mentioned in https://trac.openmodelica.org/OpenModelica/ticket/4135 were effective in this case.

I guess for the time being we should just document that OpenModelica >= 1.13 is required to avoid stability issues in generated FMUs .