Adding address sanitizer for ros2

Objective

Add a nightly job with AddressSanitizer enabled for ros2 communication layer packages.

Issue

AddressSanitizer (ASan) tool and OSRF memory tools are not working well together and resulting in test failures.
ASan runtime library replaces malloc/free and provides error reporting functions like __asan_report_load8 wiki link. On the other hand, OSRF memory tools intercept calls to dynamic memory calls like malloc/free, and provides some convenience functions for differentiating between expected and unexpected calls to dynamic memory functions.
Currently ROS2 communication tests are dependent on OSRF memory tools (dependency link) and are also setting environment variables like LD_PRELOAD to point to memory tools libraries. This prevents ASan to work properly resulting in the following error:

==17781==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD

Observations

I tried to do some local testing by manually prefixing the LD_PRELOAD value with libasan path but it resulted in test failures as follows:

2: Test command: /home/ubuntu/ros2_ws/build/osrf_testing_tools_cpp/src/test_runner/test_runner "--env" "LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libasan.so.4:/home/ubuntu/ros2_ws/build/osrf_testing_tools_cpp/src/memory_tools/libmemory_tools_interpose.so" "--" "/home/ubuntu/ros2_ws/build/osrf_testing_tools_cpp/test/memory_tools/test_memory_tools"
2: Test timeout computed to be: 1500
2: Running main() from gtest_main.cc
2: [==========] Running 2 tests from 1 test case.
2: [----------] Global test environment set-up.
2: [----------] 2 tests from TestMemoryTools
2: [ RUN      ] TestMemoryTools.test_allocation_checking_tools
2: /home/ubuntu/ros2_ws/src/osrf/osrf_testing_tools_cpp/osrf_testing_tools_cpp/test/memory_tools/test_memory_tools.cpp:95: Failure
2:       Expected: 1u
2:       Which is: 1
2: To be equal to: unexpected_mallocs
2:       Which is: 0
2: /home/ubuntu/ros2_ws/src/osrf/osrf_testing_tools_cpp/osrf_testing_tools_cpp/test/memory_tools/test_memory_tools.cpp:96: Failure
2:       Expected: 1u
2:       Which is: 1
2: To be equal to: unexpected_reallocs
2:       Which is: 0
2: /home/ubuntu/ros2_ws/src/osrf/osrf_testing_tools_cpp/osrf_testing_tools_cpp/test/memory_tools/test_memory_tools.cpp:97: Failure
2:       Expected: 1u
2:       Which is: 1
2: To be equal to: unexpected_callocs
....
....
....

Possible next steps:

Based on my observations, it looks like rmw* packages are being tested using the test_communication scripts which rely on memory tools. Is there any other way to test the rmw* packages by skipping the memory tools overrides?
One possible way to fix this would be to make address sanitizer and osrf memory tools functions compatible. I am not sure about the complexity of the same.
Does anyone know how to get unblocked about the above issue?

* Based on my observations, it looks like rmw* packages are being tested using the test_communication scripts which rely on memory tools. Is there any other way to test the rmw* packages by skipping the memory tools overrides?

We already disable memory tools on armv8 (due to a know issue), you could disable them in the same way when using ASan:

osrf_testing_tools_cpp/osrf_testing_tools_cpp/src/memory_tools/CMakeLists.txt

Lines 51 to 53 in 399a25c

    
           # TODO(wjwwood): If on aarch64, so not set the preload environment variables, until fixed. 
        
           # see: https://github.com/osrf/osrf_testing_tools_cpp/issues/3 
        
           if(NOT CMAKE_SYSTEM_PROCESSOR STREQUAL "aarch64")

* One possible way to fix this would be to make address sanitizer and osrf memory tools functions compatible. I am not sure about the complexity of the same.

My first attempt would have been to change LD_PRELOAD to be <ASan lib>:<memory_tools_lib> rather than just <memory_tools_lib>. It sounds like you tried that? If that didn't work, then I'm not sure how this might be accomplished... Nor is it clear to me that it is worth it, since we'll probably run with and without ASan (with other sanitizers and without any sanitizers as well), so I don't think it's necessarily important to have these two work together at the same time.

* Does anyone know how to get unblocked about the above issue?

I think disabling memory tools integration when using ASan is the easiest thing to do.

Thanks for the super quick response. I will experiment a bit more based on the above suggestions and get back.

Quick update on this, thanks for the tips @wjwwood. I got the sanitizer working without errors by explicitly setting the LD_PRELOAD variable to libasan path before starting the colcon tests. Here's the PR for adding a nightly address sanitizer job: ros2/ci#245.

As advised by @thomas-moulard in the above PR, I am opening an issue to add a CMake argument to disable OSRF memory testing: #23

Closing out this issue now as we are unblocked

	# TODO(wjwwood): If on aarch64, so not set the preload environment variables, until fixed.
	# see: https://github.com/osrf/osrf_testing_tools_cpp/issues/3
	if(NOT CMAKE_SYSTEM_PROCESSOR STREQUAL "aarch64")