facebookincubator/velox

Static assertion failure std::vector in QuantileDigest::validateDigest() with libstdc++ 15

kou opened this issue · 0 comments

kou commented

Problem description

Error log:

FAILED: velox/buffer/CMakeFiles/velox.dir/__/common/fuzzer/ConstrainedGenerators.cpp.o 
/bin/ccache /bin/clang++-17 -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_CONTEXT_DYN_LINK -DBOOST_CONTEXT_NO_LIB -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_FILESYSTEM_NO_LIB -DBOOST_PROGRAM_OPTIONS_DYN_LINK -DBOOST_PROGRAM_OPTIONS_NO_LIB -DBOOST_REGEX_DYN_LINK -DBOOST_REGEX_NO_LIB -DBOOST_SYSTEM_DYN_LINK -DBOOST_SYSTEM_NO_LIB -DBOOST_THREAD_DYN_LINK -DBOOST_THREAD_NO_LIB -DFMT_SHARED -DGFLAGS_IS_A_DLL=0 -DSIMDJSON_EXPERIMENTAL_ALLOW_INCOMPLETE_JSON -DSIMDJSON_THREADS_ENABLED=1 -DVELOX_ENABLE_COMPRESSION_LZ4 -DVELOX_ENABLE_PARQUET -Dvelox_EXPORTS -Ivelox/. -Ivelox/velox/external/xxhash -Ivelox.build -Ivelox/velox/tpch/gen/dbgen/include -Ivelox/velox/tpcds/gen/dsdgen/include -Ivelox.build/_deps/xsimd-src/include -Ivelox.build/_deps/simdjson-src/include -isystem velox/velox -isystem velox/velox/external -isystem /usr/include/libdwarf -isystem /usr/include/libiberty -isystem /var/tmp/local.velox/include -isystem /usr/include/double-conversion -isystem velox.build/CMake/resolve_dependency_modules/arrow/arrow_ep/install/include -mavx2 -mfma -mavx -mf16c -mlzcnt -mbmi2 -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -DFOLLY_CFG_NO_COROUTINES -Wall -Wextra -Wno-unused        -Wno-unused-parameter        -Wno-sign-compare        -Wno-ignored-qualifiers        -Wno-range-loop-analysis          -Wno-mismatched-tags          -Wno-nullability-completeness -g -std=gnu++20 -fPIC -pthread -DNOMINMAX -MD -MT velox/buffer/CMakeFiles/velox.dir/__/common/fuzzer/ConstrainedGenerators.cpp.o -MF velox/buffer/CMakeFiles/velox.dir/__/common/fuzzer/ConstrainedGenerators.cpp.o.d @velox/buffer/CMakeFiles/velox.dir/__/common/fuzzer/ConstrainedGenerators.cpp.o.modmap -o velox/buffer/CMakeFiles/velox.dir/__/common/fuzzer/ConstrainedGenerators.cpp.o -c velox/velox/common/fuzzer/ConstrainedGenerators.cpp
In file included from velox/velox/common/fuzzer/ConstrainedGenerators.cpp:17:
In file included from velox/./velox/common/fuzzer/ConstrainedGenerators.h:21:
In file included from /var/tmp/local.velox/include/folly/json.h:17:
In file included from /var/tmp/local.velox/include/folly/json/json.h:47:
In file included from /var/tmp/local.velox/include/folly/Function.h:204:
In file included from /bin/../lib/gcc/x86_64-linux-gnu/15/../../../../include/c++/15/functional:66:
In file included from /bin/../lib/gcc/x86_64-linux-gnu/15/../../../../include/c++/15/vector:69:
/bin/../lib/gcc/x86_64-linux-gnu/15/../../../../include/c++/15/bits/stl_bvector.h:755:21: error: static assertion failed due to requirement 'is_same<unsigned long, bool>::value': std::vector must have the same value_type as its allocator
  755 |       static_assert(is_same<typename _Alloc::value_type, bool>::value,
      |                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
velox/./velox/functions/lib/QuantileDigest.h:779:49: note: in instantiation of template class 'std::vector<bool, std::allocator<unsigned long>>' requested here
  779 |   std::vector<bool, RebindAlloc<unsigned long>> visited(
      |                                                 ^
velox/./velox/functions/lib/QuantileDigest.h:838:16: note: in instantiation of member function 'facebook::velox::functions::qdigest::QuantileDigest<long, std::allocator<long>>::validateDigest' requested here
  838 |   VELOX_DCHECK(validateDigest());
      |                ^
velox/./velox/functions/lib/QuantileDigest.h:612:5: note: in instantiation of member function 'facebook::velox::functions::qdigest::QuantileDigest<long, std::allocator<long>>::compress' requested here
  612 |     compress();
      |     ^
velox/./velox/common/fuzzer/ConstrainedGenerators.h:587:14: note: in instantiation of member function 'facebook::velox::functions::qdigest::QuantileDigest<long, std::allocator<long>>::add' requested here
  587 |       digest.add(value, dist(rng_));
      |              ^
velox/velox/common/fuzzer/ConstrainedGenerators.cpp:337:16: note: in instantiation of function template specialization 'facebook::velox::fuzzer::QDigestInputGenerator::createSerializedDigest<long>' requested here
  337 |         return createSerializedDigest<int64_t>(len, kAccuracy);
      |                ^
In file included from velox/velox/common/fuzzer/ConstrainedGenerators.cpp:17:
In file included from velox/./velox/common/fuzzer/ConstrainedGenerators.h:24:

I think that this only happens with libstdc++ 15 or later.

This is caused by #13772 . It uses different types (bool and unsigned long) for std::vector<>:

std::vector<bool, RebindAlloc<unsigned long>> visited(
lefts_.size(), false, RebindAlloc<unsigned long>(lefts_.get_allocator()));

If we use the same type (bool) for them, we can fix the static assertion:

diff --git a/velox/functions/lib/QuantileDigest.h b/velox/functions/lib/QuantileDigest.h
index 2b117346c..ca9ded610 100644
--- a/velox/functions/lib/QuantileDigest.h
+++ b/velox/functions/lib/QuantileDigest.h
@@ -776,8 +776,8 @@ bool QuantileDigest<T, Allocator>::validateDigest() const {
     free.insert(iterator);
     iterator = lefts_[iterator];
   }
-  std::vector<bool, RebindAlloc<unsigned long>> visited(
-      lefts_.size(), false, RebindAlloc<unsigned long>(lefts_.get_allocator()));
+  std::vector<bool, RebindAlloc<bool>> visited(
+      lefts_.size(), false, RebindAlloc<bool>(lefts_.get_allocator()));
 
   // Check that visited nodes are not in the free list and are visited only
   // once.

System information

Velox System Info v0.0.2
Commit: eac4b4b
CMake Version: 3.31.6
System: Linux-6.16.3+deb14-amd64
Arch: x86_64
C++ Compiler: /bin/c++
C++ Compiler Version: 15.2.0
C Compiler: /bin/cc
C Compiler Version: 15.2.0
CMake Prefix Path: /usr/local;/usr;/;/usr;/usr/local;/usr/X11R6;/usr/pkg;/opt

CMake log