pisa-engine/pisa

Address sanitizer error in BMW queries test

elshize opened this issue · 0 comments

elshize commented

Describe the bug

With address sanitizer on and on Clang 15, test_bmw_queries fails.

To Reproduce

Steps to reproduce the behavior:

  1. Compile with Clang 15 (and libc++) and -DUSE_SANITIZERS=ON
  2. Run test_bmw_queries test.

Error message

=================================================================
==2939095==ERROR: AddressSanitizer: container-overflow on address 0x62d000026258 at pc 0x0000007f5837 bp 0x7ffcf8db2e50 sp 0x7ffcf8db2e48
READ of size 4 at 0x62d000026258 thread T0
    #0 0x7f5836 in pisa::score_opt_partition::score_window<std::__1::__wrap_iter<std::__1::pair<unsigned long, float>*>>::advance_end() /home/elshize/dev/pisa/include/pisa/score_opt_partition.hpp:88:17
    #1 0x7f5836 in pisa::score_opt_partition::score_opt_partition<std::__1::__wrap_iter<std::__1::pair<unsigned long, float>*>>(std::__1::__wrap_iter<std::__1::pair<unsigned long, float>*>, unsigned int, unsigned long, double, double, float) /home/elshize/dev/pisa/include/pisa/score_opt_partition.hpp:167:28
    #2 0x7ec72d in std::__1::pair<std::__1::vector<unsigned int, std::__1::allocator<unsigned int>>, std::__1::vector<float, std::__1::allocator<float>>> pisa::variable_block_partition<std::__1::function<float (unsigned int, unsigned int)>>(pisa::binary_freq_collection const&, pisa::binary_freq_collection::sequence const&, std::__1::function<float (unsigned int, unsigned int)>, float, double, double) /home/elshize/dev/pisa/include/pisa/wand_utils.hpp:80:14
    #3 0x7e86c2 in float pisa::wand_data_raw::builder::add_sequence<std::__1::function<float (unsigned int, unsigned int)>>(pisa::binary_freq_collection::sequence const&, pisa::binary_freq_collection const&, std::__1::vector<unsigned int, std::__1::allocator<unsigned int>> const&, float, std::__1::function<float (unsigned int, unsigned int)>, boost::variant<pisa::FixedBlock, pisa::VariableBlock>) /home/elshize/dev/pisa/include/pisa/wand_data_raw.hpp:44:19
    #4 0x740319 in pisa::wand_data<pisa::wand_data_raw>::wand_data<unsigned int const*>(unsigned int const*, unsigned long, pisa::binary_freq_collection const&, ScorerParams const&, boost::variant<pisa::FixedBlock, pisa::VariableBlock>, bool, std::__1::unordered_set<unsigned long, std::__1::hash<unsigned long>, std::__1::equal_to<unsigned long>, std::__1::allocator<unsigned long>> const&) /home/elshize/dev/pisa/include/pisa/wand_data.hpp:98:34
    #5 0x74d4b2 in IndexData<pisa::freq_index<pisa::indexed_sequence, pisa::positive_sequence<pisa::strict_sequence>>>::IndexData(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::unordered_set<unsigned long, std::__1::hash<unsigned long>, std::__1::equal_to<unsigned long>, std::__1::allocator<unsigned long>> const&) /home/elshize/dev/pisa/test/test_bmw_queries.cpp:31:11
    #6 0x739d1f in std::__1::__unique_if<IndexData<pisa::freq_index<pisa::indexed_sequence, pisa::positive_sequence<pisa::strict_sequence>>>>::__unique_single std::__1::make_unique[abi:v15007]<IndexData<pisa::freq_index<pisa::indexed_sequence, pisa::positive_sequence<pisa::strict_sequence>>>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::unordered_set<unsigned long, std::__1::hash<unsigned long>, std::__1::equal_to<unsigned long>, std::__1::allocator<unsigned long>> const&>(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::unordered_set<unsigned long, std::__1::hash<unsigned long>, std::__1::equal_to<unsigned long>, std::__1::allocator<unsigned long>> const&) /usr/bin/../include/c++/v1/__memory/unique_ptr.h:714:32
    #7 0x739d1f in IndexData<pisa::freq_index<pisa::indexed_sequence, pisa::positive_sequence<pisa::strict_sequence>>>::get(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::unordered_set<unsigned long, std::__1::hash<unsigned long>, std::__1::equal_to<unsigned long>, std::__1::allocator<unsigned long>> const&) /home/elshize/dev/pisa/test/test_bmw_queries.cpp:67:39
    #8 0x65e011 in ____C_A_T_C_H____T_E_S_T____0() /home/elshize/dev/pisa/test/test_bmw_queries.cpp:110:21
    #9 0x619ec7 in Catch::TestCase::invoke() const /home/elshize/dev/pisa/external/Catch2/single_include/catch2/catch.hpp:14160:15
    #10 0x619ec7 in Catch::RunContext::invokeActiveTestCase() /home/elshize/dev/pisa/external/Catch2/single_include/catch2/catch.hpp:13020:27
    #11 0x61790c in Catch::RunContext::runCurrentTest(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&) /home/elshize/dev/pisa/external/Catch2/single_include/catch2/catch.hpp:12993:17
    #12 0x6156de in Catch::RunContext::runTest(Catch::TestCase const&) /home/elshize/dev/pisa/external/Catch2/single_include/catch2/catch.hpp:12754:13
    #13 0x621725 in Catch::(anonymous namespace)::TestGroup::execute() /home/elshize/dev/pisa/external/Catch2/single_include/catch2/catch.hpp:13347:45
    #14 0x621725 in Catch::Session::runInternal() /home/elshize/dev/pisa/external/Catch2/single_include/catch2/catch.hpp:13553:39
    #15 0x61f057 in Catch::Session::run() /home/elshize/dev/pisa/external/Catch2/single_include/catch2/catch.hpp:13509:24
    #16 0x65d419 in int Catch::Session::run<char>(int, char const* const*) /home/elshize/dev/pisa/external/Catch2/single_include/catch2/catch.hpp:13231:30
    #17 0x65d419 in main /home/elshize/dev/pisa/external/Catch2/single_include/catch2/catch.hpp:17526:29
    #18 0x7f315b64c50f in __libc_start_call_main (/usr/lib64/libc.so.6+0x2750f) (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
    #19 0x7f315b64c5c8 in __libc_start_main@GLIBC_2.2.5 (/usr/lib64/libc.so.6+0x275c8) (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
    #20 0x4a6384 in _start (/home/elshize/dev/pisa/build/test/test_bmw_queries+0x4a6384) (BuildId: cd726b00204df488dfd10996a40d1e86e5e213dd)

0x62d000026258 is located 32344 bytes inside of 32768-byte region [0x62d00001e400,0x62d000026400)
allocated by thread T0 here:
    #0 0x59e9b8 in operator new(unsigned long) (/home/elshize/dev/pisa/build/test/test_bmw_queries+0x59e9b8) (BuildId: cd726b00204df488dfd10996a40d1e86e5e213dd)
    #1 0x7f6dda in void* std::__1::__libcpp_operator_new[abi:v15007]<unsigned long>(unsigned long) /usr/bin/../include/c++/v1/new:246:10
    #2 0x7f6dda in std::__1::__libcpp_allocate[abi:v15007](unsigned long, unsigned long) /usr/bin/../include/c++/v1/new:272:10
    #3 0x7f6dda in std::__1::allocator<std::__1::pair<unsigned long, float>>::allocate[abi:v15007](unsigned long) /usr/bin/../include/c++/v1/__memory/allocator.h:112:38
    #4 0x7f6dda in std::__1::__allocation_result<std::__1::allocator_traits<std::__1::allocator<std::__1::pair<unsigned long, float>>>::pointer> std::__1::__allocate_at_least[abi:v15007]<std::__1::allocator<std::__1::pair<unsigned long, float>>>(std::__1::allocator<std::__1::pair<unsigned long, float>>&, unsigned long) /usr/bin/../include/c++/v1/__memory/allocate_at_least.h:54:19
    #5 0x7f6dda in std::__1::__split_buffer<std::__1::pair<unsigned long, float>, std::__1::allocator<std::__1::pair<unsigned long, float>>&>::__split_buffer(unsigned long, unsigned long, std::__1::allocator<std::__1::pair<unsigned long, float>>&) /usr/bin/../include/c++/v1/__split_buffer:316:29
    #6 0x7f6dda in void std::__1::vector<std::__1::pair<unsigned long, float>, std::__1::allocator<std::__1::pair<unsigned long, float>>>::__push_back_slow_path<std::__1::pair<unsigned long, float>>(std::__1::pair<unsigned long, float>&&) /usr/bin/../include/c++/v1/vector:1569:49
    #7 0x7ec45c in std::__1::vector<std::__1::pair<unsigned long, float>, std::__1::allocator<std::__1::pair<unsigned long, float>>>::push_back[abi:v15007](std::__1::pair<unsigned long, float>&&) /usr/bin/../include/c++/v1/vector:1601:9
    #8 0x7ec45c in std::__1::back_insert_iterator<std::__1::vector<std::__1::pair<unsigned long, float>, std::__1::allocator<std::__1::pair<unsigned long, float>>>>::operator=[abi:v15007](std::__1::pair<unsigned long, float>&&) /usr/bin/../include/c++/v1/__iterator/back_insert_iterator.h:53:21
    #9 0x7ec45c in std::__1::back_insert_iterator<std::__1::vector<std::__1::pair<unsigned long, float>, std::__1::allocator<std::__1::pair<unsigned long, float>>>> std::__1::transform[abi:v15007]<unsigned int const*, unsigned int const*, std::__1::back_insert_iterator<std::__1::vector<std::__1::pair<unsigned long, float>, std::__1::allocator<std::__1::pair<unsigned long, float>>>>, std::__1::pair<std::__1::vector<unsigned int, std::__1::allocator<unsigned int>>, std::__1::vector<float, std::__1::allocator<float>>> pisa::variable_block_partition<std::__1::function<float (unsigned int, unsigned int)>>(pisa::binary_freq_collection const&, pisa::binary_freq_collection::sequence const&, std::__1::function<float (unsigned int, unsigned int)>, float, double, double)::'lambda'(unsigned long const&, unsigned long const&)>(std::__1::function<float (unsigned int, unsigned int)>, std::__1::function<float (unsigned int, unsigned int)>, unsigned int const*, std::__1::back_insert_iterator<std::__1::vector<std::__1::pair<unsigned long, float>, std::__1::allocator<std::__1::pair<unsigned long, float>>>>, std::__1::pair<std::__1::vector<unsigned int, std::__1::allocator<unsigned int>>, std::__1::vector<float, std::__1::allocator<float>>> pisa::variable_block_partition<std::__1::function<float (unsigned int, unsigned int)>>(pisa::binary_freq_collection const&, pisa::binary_freq_collection::sequence const&, std::__1::function<float (unsigned int, unsigned int)>, float, double, double)::'lambda'(unsigned long const&, unsigned long const&)) /usr/bin/../include/c++/v1/__algorithm/transform.h:37:19
    #10 0x7ec45c in std::__1::pair<std::__1::vector<unsigned int, std::__1::allocator<unsigned int>>, std::__1::vector<float, std::__1::allocator<float>>> pisa::variable_block_partition<std::__1::function<float (unsigned int, unsigned int)>>(pisa::binary_freq_collection const&, pisa::binary_freq_collection::sequence const&, std::__1::function<float (unsigned int, unsigned int)>, float, double, double) /home/elshize/dev/pisa/include/pisa/wand_utils.hpp:71:5
    #11 0x7e86c2 in float pisa::wand_data_raw::builder::add_sequence<std::__1::function<float (unsigned int, unsigned int)>>(pisa::binary_freq_collection::sequence const&, pisa::binary_freq_collection const&, std::__1::vector<unsigned int, std::__1::allocator<unsigned int>> const&, float, std::__1::function<float (unsigned int, unsigned int)>, boost::variant<pisa::FixedBlock, pisa::VariableBlock>) /home/elshize/dev/pisa/include/pisa/wand_data_raw.hpp:44:19
    #12 0x740319 in pisa::wand_data<pisa::wand_data_raw>::wand_data<unsigned int const*>(unsigned int const*, unsigned long, pisa::binary_freq_collection const&, ScorerParams const&, boost::variant<pisa::FixedBlock, pisa::VariableBlock>, bool, std::__1::unordered_set<unsigned long, std::__1::hash<unsigned long>, std::__1::equal_to<unsigned long>, std::__1::allocator<unsigned long>> const&) /home/elshize/dev/pisa/include/pisa/wand_data.hpp:98:34
    #13 0x74d4b2 in IndexData<pisa::freq_index<pisa::indexed_sequence, pisa::positive_sequence<pisa::strict_sequence>>>::IndexData(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::unordered_set<unsigned long, std::__1::hash<unsigned long>, std::__1::equal_to<unsigned long>, std::__1::allocator<unsigned long>> const&) /home/elshize/dev/pisa/test/test_bmw_queries.cpp:31:11
    #14 0x739d1f in std::__1::__unique_if<IndexData<pisa::freq_index<pisa::indexed_sequence, pisa::positive_sequence<pisa::strict_sequence>>>>::__unique_single std::__1::make_unique[abi:v15007]<IndexData<pisa::freq_index<pisa::indexed_sequence, pisa::positive_sequence<pisa::strict_sequence>>>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::unordered_set<unsigned long, std::__1::hash<unsigned long>, std::__1::equal_to<unsigned long>, std::__1::allocator<unsigned long>> const&>(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::unordered_set<unsigned long, std::__1::hash<unsigned long>, std::__1::equal_to<unsigned long>, std::__1::allocator<unsigned long>> const&) /usr/bin/../include/c++/v1/__memory/unique_ptr.h:714:32
    #15 0x739d1f in IndexData<pisa::freq_index<pisa::indexed_sequence, pisa::positive_sequence<pisa::strict_sequence>>>::get(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::unordered_set<unsigned long, std::__1::hash<unsigned long>, std::__1::equal_to<unsigned long>, std::__1::allocator<unsigned long>> const&) /home/elshize/dev/pisa/test/test_bmw_queries.cpp:67:39
    #16 0x65e011 in ____C_A_T_C_H____T_E_S_T____0() /home/elshize/dev/pisa/test/test_bmw_queries.cpp:110:21
    #17 0x619ec7 in Catch::TestCase::invoke() const /home/elshize/dev/pisa/external/Catch2/single_include/catch2/catch.hpp:14160:15
    #18 0x619ec7 in Catch::RunContext::invokeActiveTestCase() /home/elshize/dev/pisa/external/Catch2/single_include/catch2/catch.hpp:13020:27
    #19 0x61790c in Catch::RunContext::runCurrentTest(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&) /home/elshize/dev/pisa/external/Catch2/single_include/catch2/catch.hpp:12993:17
    #20 0x6156de in Catch::RunContext::runTest(Catch::TestCase const&) /home/elshize/dev/pisa/external/Catch2/single_include/catch2/catch.hpp:12754:13
    #21 0x621725 in Catch::(anonymous namespace)::TestGroup::execute() /home/elshize/dev/pisa/external/Catch2/single_include/catch2/catch.hpp:13347:45
    #22 0x621725 in Catch::Session::runInternal() /home/elshize/dev/pisa/external/Catch2/single_include/catch2/catch.hpp:13553:39
    #23 0x61f057 in Catch::Session::run() /home/elshize/dev/pisa/external/Catch2/single_include/catch2/catch.hpp:13509:24
    #24 0x65d419 in int Catch::Session::run<char>(int, char const* const*) /home/elshize/dev/pisa/external/Catch2/single_include/catch2/catch.hpp:13231:30
    #25 0x65d419 in main /home/elshize/dev/pisa/external/Catch2/single_include/catch2/catch.hpp:17526:29
    #26 0x7f315b64c50f in __libc_start_call_main (/usr/lib64/libc.so.6+0x2750f) (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)

HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_container_overflow=0.
If you suspect a false positive see also: https://github.com/google/sanitizers/wiki/AddressSanitizerContainerOverflow.
SUMMARY: AddressSanitizer: container-overflow /home/elshize/dev/pisa/include/pisa/score_opt_partition.hpp:88:17 in pisa::score_opt_partition::score_window<std::__1::__wrap_iter<std::__1::pair<unsigned long, float>*>>::advance_end()
Shadow bytes around the buggy address:
  0x0c5a7fffcbf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c5a7fffcc00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c5a7fffcc10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c5a7fffcc20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c5a7fffcc30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c5a7fffcc40: 00 00 00 00 00 00 00 00 00 00 fc[fc]fc fc fc fc
  0x0c5a7fffcc50: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
  0x0c5a7fffcc60: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
  0x0c5a7fffcc70: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
  0x0c5a7fffcc80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c5a7fffcc90: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==2939095==ABORTING

Environment info

Operating System: Fedora 37
Compiler name: Clang
Compiler version: 15