dev-libs/apache-arrow: So much badness in "apache-arrow-9.0.0-r1"
leycec opened this issue · 2 comments
leycec commented
Gah! Apache Arrow no longer builds with the default +lz4 USE flag enabled, which is pretty weird. I'm certain that it used to build with this flag enabled. But... the Portage logfile doesn't lie:
$ emerge apache-arrow
These are the packages that would be merged:
Calculating dependencies ... done!
[ebuild R ~] dev-libs/apache-arrow-9.0.0-r1::spark-overlay USE="bzip2 lz4 parquet python zlib zstd" PYTHON_SINGLE_TARGET="python3_9 -python3_7 -python3_8" 0 KiB
Total: 1 package (1 reinstall), Size of downloads: 0 KiB
Would you like to merge these packages? [Yes/No] Yes
>>> Verifying ebuild manifests
>>> Emerging (1 of 1) dev-libs/apache-arrow-9.0.0-r1::spark-overlay
* apache-arrow-9.0.0.tar.gz BLAKE2B SHA512 size ;-) ... [ ok ]
* Using python3.10 to build
>>> Unpacking source...
>>> Unpacking apache-arrow-9.0.0.tar.gz to /var/tmp/portage/dev-libs/apache-arrow-9.0.0/work
>>> Source unpacked in /var/tmp/portage/dev-libs/apache-arrow-9.0.0/work
>>> Preparing source in /var/tmp/portage/dev-libs/apache-arrow-9.0.0/work/apache-arrow-9.0.0/cpp ...
* Source directory (CMAKE_USE_DIR): "/var/tmp/portage/dev-libs/apache-arrow-9.0.0/work/apache-arrow-9.0.0/cpp"
* Build directory (BUILD_DIR): "/var/tmp/portage/dev-libs/apache-arrow-9.0.0/work/apache-arrow-9.0.0/cpp_build"
* Applying apache-arrow-9.0.0-thrift-limit.patch ... [ ok ]
* Hardcoded definition(s) removed in examples/minimal_build/CMakeLists.txt:
* set(CMAKE_BUILD_TYPE Release)
>>> Source prepared.
>>> Configuring source in /var/tmp/portage/dev-libs/apache-arrow-9.0.0/work/apache-arrow-9.0.0/cpp ...
* Source directory (CMAKE_USE_DIR): "/var/tmp/portage/dev-libs/apache-arrow-9.0.0/work/apache-arrow-9.0.0/cpp"
* Build directory (BUILD_DIR): "/var/tmp/portage/dev-libs/apache-arrow-9.0.0/work/apache-arrow-9.0.0/cpp_build"
cmake -C /var/tmp/portage/dev-libs/apache-arrow-9.0.0/work/apache-arrow-9.0.0/cpp_build/gentoo_common_config.cmake -G Ninja -DCMAKE_INSTALL_PREFIX=/usr -DARROW_CXXFLAGS=-DNDEBUG -DARROW_DEPENDENCY_SOURCE=SYSTEM -DARROW_BUILD_STATIC=OFF -DARROW_DOC_DIR=share/doc/apache-arrow-9.0.0 -DARROW_JEMALLOC=OFF -DARROW_PARQUET=ON -DARROW_WITH_BZ2=ON -DARROW_WITH_LZ4=ON -DARROW_PYTHON=ON -DARROW_WITH_ZLIB=ON -DARROW_WITH_ZSTD=ON -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_TOOLCHAIN_FILE=/var/tmp/portage/dev-libs/apache-arrow-9.0.0/work/apache-arrow-9.0.0/cpp_build/gentoo_toolchain.cmake /var/tmp/portage/dev-libs/apache-arrow-9.0.0/work/apache-arrow-9.0.0/cpp
loading initial cache file /var/tmp/portage/dev-libs/apache-arrow-9.0.0/work/apache-arrow-9.0.0/cpp_build/gentoo_common_config.cmake
-- Building using CMake version: 3.24.2
-- The C compiler identification is GNU 11.3.0
-- The CXX compiler identification is GNU 11.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/x86_64-pc-linux-gnu-gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/x86_64-pc-linux-gnu-g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Arrow version: 9.0.0 (full: '9.0.0')
-- Arrow SO version: 900 (full: 900.0.0)
-- clang-tidy 12 not found
-- clang-format 12 not found
-- Could NOT find ClangTools (missing: CLANG_FORMAT_BIN CLANG_TIDY_BIN)
-- infer not found
fatal: not a git repository (or any parent up to mount point /var/tmp)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
-- Found Python3: /var/tmp/portage/dev-libs/apache-arrow-9.0.0/temp/python3.10/bin/python3 (found version "3.10.8") found components: Interpreter
-- Using ccache: /usr/bin/ccache
-- Found cpplint executable at /var/tmp/portage/dev-libs/apache-arrow-9.0.0/work/apache-arrow-9.0.0/cpp/build-support/cpplint.py
-- Performing Test CXX_LINKER_SUPPORTS_VERSION_SCRIPT
-- Performing Test CXX_LINKER_SUPPORTS_VERSION_SCRIPT - Success
-- Using SYSTEM approach to find dependencies
-- ARROW_ABSL_BUILD_VERSION: 20211102.0
-- ARROW_ABSL_BUILD_SHA256_CHECKSUM: dcf71b9cba8dc0ca9940c4b316a0c796be8fab42b070bb6b7cab62b48f0e66c4
-- ARROW_AWSSDK_BUILD_VERSION: 1.8.133
-- ARROW_AWSSDK_BUILD_SHA256_CHECKSUM: d6c495bc06be5e21dac716571305d77437e7cfd62a2226b8fe48d9ab5785a8d6
-- ARROW_AWS_CHECKSUMS_BUILD_VERSION: v0.1.12
-- ARROW_AWS_CHECKSUMS_BUILD_SHA256_CHECKSUM: 394723034b81cc7cd528401775bc7aca2b12c7471c92350c80a0e2fb9d2909fe
-- ARROW_AWS_C_COMMON_BUILD_VERSION: v0.6.9
-- ARROW_AWS_C_COMMON_BUILD_SHA256_CHECKSUM: 928a3e36f24d1ee46f9eec360ec5cebfe8b9b8994fe39d4fa74ff51aebb12717
-- ARROW_AWS_C_EVENT_STREAM_BUILD_VERSION: v0.1.5
-- ARROW_AWS_C_EVENT_STREAM_BUILD_SHA256_CHECKSUM: f1b423a487b5d6dca118bfc0d0c6cc596dc476b282258a3228e73a8f730422d4
-- ARROW_BOOST_BUILD_VERSION: 1.75.0
-- ARROW_BOOST_BUILD_SHA256_CHECKSUM: 267e04a7c0bfe85daf796dedc789c3a27a76707e1c968f0a2a87bb96331e2b61
-- ARROW_BROTLI_BUILD_VERSION: v1.0.9
-- ARROW_BROTLI_BUILD_SHA256_CHECKSUM: f9e8d81d0405ba66d181529af42a3354f838c939095ff99930da6aa9cdf6fe46
-- ARROW_BZIP2_BUILD_VERSION: 1.0.8
-- ARROW_BZIP2_BUILD_SHA256_CHECKSUM: ab5a03176ee106d3f0fa90e381da478ddae405918153cca248e682cd0c4a2269
-- ARROW_CARES_BUILD_VERSION: 1.17.2
-- ARROW_CARES_BUILD_SHA256_CHECKSUM: 4803c844ce20ce510ef0eb83f8ea41fa24ecaae9d280c468c582d2bb25b3913d
-- ARROW_CRC32C_BUILD_VERSION: 1.1.2
-- ARROW_CRC32C_BUILD_SHA256_CHECKSUM: ac07840513072b7fcebda6e821068aa04889018f24e10e46181068fb214d7e56
-- ARROW_GBENCHMARK_BUILD_VERSION: v1.6.0
-- ARROW_GBENCHMARK_BUILD_SHA256_CHECKSUM: 1f71c72ce08d2c1310011ea6436b31e39ccab8c2db94186d26657d41747c85d6
-- ARROW_GFLAGS_BUILD_VERSION: v2.2.2
-- ARROW_GFLAGS_BUILD_SHA256_CHECKSUM: 34af2f15cf7367513b352bdcd2493ab14ce43692d2dcd9dfc499492966c64dcf
-- ARROW_GLOG_BUILD_VERSION: v0.5.0
-- ARROW_GLOG_BUILD_SHA256_CHECKSUM: eede71f28371bf39aa69b45de23b329d37214016e2055269b3b5e7cfd40b59f5
-- ARROW_GOOGLE_CLOUD_CPP_BUILD_VERSION: v1.42.0
-- ARROW_GOOGLE_CLOUD_CPP_BUILD_SHA256_CHECKSUM: c06ae9aededbb8aa217a6d2453754daa40b815f9a4004bc4f2d2d215c79828aa
-- ARROW_GRPC_BUILD_VERSION: v1.46.3
-- ARROW_GRPC_BUILD_SHA256_CHECKSUM: d6cbf22cb5007af71b61c6be316a79397469c58c82a942552a62e708bce60964
-- ARROW_GTEST_BUILD_VERSION: 1.11.0
-- ARROW_GTEST_BUILD_SHA256_CHECKSUM: b4870bf121ff7795ba20d20bcdd8627b8e088f2d1dab299a031c1034eddc93d5
-- ARROW_JEMALLOC_BUILD_VERSION: 5.3.0
-- ARROW_JEMALLOC_BUILD_SHA256_CHECKSUM: 2db82d1e7119df3e71b7640219b6dfe84789bc0537983c3b7ac4f7189aecfeaa
-- ARROW_LZ4_BUILD_VERSION: 8f61d8eb7c6979769a484cde8df61ff7c4c77765
-- ARROW_LZ4_BUILD_SHA256_CHECKSUM: 5b072d848f2f93fddb97e1143e22c1bd7fa19fc8431ee69e21758190a88125b4
-- ARROW_MIMALLOC_BUILD_VERSION: v2.0.6
-- ARROW_MIMALLOC_BUILD_SHA256_CHECKSUM: 9f05c94cc2b017ed13698834ac2a3567b6339a8bde27640df5a1581d49d05ce5
-- ARROW_NLOHMANN_JSON_BUILD_VERSION: v3.10.5
-- ARROW_NLOHMANN_JSON_BUILD_SHA256_CHECKSUM: 5daca6ca216495edf89d167f808d1d03c4a4d929cef7da5e10f135ae1540c7e4
-- ARROW_OPENTELEMETRY_BUILD_VERSION: v1.4.1
-- ARROW_OPENTELEMETRY_BUILD_SHA256_CHECKSUM: 301b1ab74a664723560f46c29f228360aff1e2d63e930b963755ea077ae67524
-- ARROW_OPENTELEMETRY_PROTO_BUILD_VERSION: v0.17.0
-- ARROW_OPENTELEMETRY_PROTO_BUILD_SHA256_CHECKSUM: f269fbcb30e17b03caa1decd231ce826e59d7651c0f71c3b28eb5140b4bb5412
-- ARROW_ORC_BUILD_VERSION: 1.7.5
-- ARROW_ORC_BUILD_SHA256_CHECKSUM: b90cae5853e3ea0e5bf64e3883d09d871ec280536e0bb2cc52d08f79f0be0010
-- ARROW_PROTOBUF_BUILD_VERSION: v21.3
-- ARROW_PROTOBUF_BUILD_SHA256_CHECKSUM: 2f723218f6cb709ae4cdc4fb5ed56a5951fc5d466f0128ce4c946b8c78c8c49f
-- ARROW_RAPIDJSON_BUILD_VERSION: 232389d4f1012dddec4ef84861face2d2ba85709
-- ARROW_RAPIDJSON_BUILD_SHA256_CHECKSUM: b9290a9a6d444c8e049bd589ab804e0ccf2b05dc5984a19ed5ae75d090064806
-- ARROW_RE2_BUILD_VERSION: 2022-06-01
-- ARROW_RE2_BUILD_SHA256_CHECKSUM: f89c61410a072e5cbcf8c27e3a778da7d6fd2f2b5b1445cd4f4508bee946ab0f
-- ARROW_SNAPPY_BUILD_VERSION: 1.1.9
-- ARROW_SNAPPY_BUILD_SHA256_CHECKSUM: 75c1fbb3d618dd3a0483bff0e26d0a92b495bbe5059c8b4f1c962b478b6e06e7
-- ARROW_SNAPPY_OLD_BUILD_VERSION: 1.1.8
-- ARROW_SNAPPY_OLD_BUILD_SHA256_CHECKSUM: 16b677f07832a612b0836178db7f374e414f94657c138e6993cbfc5dcc58651f
-- ARROW_SUBSTRAIT_BUILD_VERSION: v0.6.0
-- ARROW_SUBSTRAIT_BUILD_SHA256_CHECKSUM: 7b8583b9684477e9027f417bbfb4febb8acfeb01923dcaa7cf0fd3f921d69c88
-- ARROW_THRIFT_BUILD_VERSION: 0.16.0
-- ARROW_THRIFT_BUILD_SHA256_CHECKSUM: f460b5c1ca30d8918ff95ea3eb6291b3951cf518553566088f3f2be8981f6209
-- ARROW_UCX_BUILD_VERSION: 1.12.1
-- ARROW_UCX_BUILD_SHA256_CHECKSUM: 9bef31aed0e28bf1973d28d74d9ac4f8926c43ca3b7010bd22a084e164e31b71
-- ARROW_UTF8PROC_BUILD_VERSION: v2.7.0
-- ARROW_UTF8PROC_BUILD_SHA256_CHECKSUM: 4bb121e297293c0fd55f08f83afab6d35d48f0af4ecc07523ad8ec99aa2b12a1
-- ARROW_XSIMD_BUILD_VERSION: 8.1.0
-- ARROW_XSIMD_BUILD_SHA256_CHECKSUM: d52551360d37709675237d2a0418e28f70995b5b7cdad7c674626bcfbbf48328
-- ARROW_ZLIB_BUILD_VERSION: 1.2.12
-- ARROW_ZLIB_BUILD_SHA256_CHECKSUM: 91844808532e5ce316b3c010929493c0244f3d37593afd6de04f71821d5136d9
-- ARROW_ZSTD_BUILD_VERSION: v1.5.2
-- ARROW_ZSTD_BUILD_SHA256_CHECKSUM: f7de13462f7a82c29ab865820149e778cbfe01087b3a55b5332707abf9db4a6e
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Looking for _M_ARM64
-- Looking for _M_ARM64 - not found
-- Looking for __SIZEOF_INT128__
-- Looking for __SIZEOF_INT128__ - found
-- Found Boost: /usr/lib64/cmake/Boost-1.79.0/BoostConfig.cmake (found suitable version "1.79.0", minimum required is "1.58") found components: system filesystem
-- Boost include dir: /usr/include
-- Building without OpenSSL support. Minimum OpenSSL version 1.0.2 required.
-- Checking for module 'thrift'
-- Package 'thrift', required by 'virtual:world', not found
-- Found Thrift: /usr/lib64/libthrift.so (found suitable version "0.16.0", minimum required is "0.11.0")
-- Providing CMake module for Thrift
-- RapidJSON found. Headers: /usr/include
-- xsimd found. Headers: /usr/include
-- Found ZLIB: /usr/lib64/libz.so (found version "1.2.12")
-- Providing CMake module for lz4Alt
-- Checking for module 'libzstd'
-- Found libzstd, version 1.5.2
-- Found zstd: /usr/lib64/libzstd.so (Required is at least version "1.4.0")
-- Providing CMake module for zstd
CMake Warning at cmake_modules/Findre2Alt.cmake:25 (find_package):
By not providing "Findre2.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "re2", but
CMake did not find one.
Could not find a package configuration file provided by "re2" with any of
the following names:
re2Config.cmake
re2-config.cmake
Add the installation prefix of "re2" to CMAKE_PREFIX_PATH or set "re2_DIR"
to a directory containing one of the above files. If "re2" provides a
separate development package or SDK, be sure it has been installed.
Call Stack (most recent call first):
cmake_modules/ThirdpartyToolchain.cmake:267 (find_package)
cmake_modules/ThirdpartyToolchain.cmake:2482 (resolve_dependency)
CMakeLists.txt:573 (include)
-- Checking for module 're2'
-- Found re2, version 0.0.0
-- Found re2Alt: /usr/lib64/libre2.so
-- Providing CMake module for re2Alt
-- Found BZip2: /usr/lib64/libbz2.so (found version "1.0.8")
-- Looking for BZ2_bzCompressInit
-- Looking for BZ2_bzCompressInit - found
-- Found utf8proc: /usr/lib64/libutf8proc.so (found suitable version "2.7.0", minimum required is "2.2.0")
-- Providing CMake module for utf8proc
-- Found hdfs.h at: /var/tmp/portage/dev-libs/apache-arrow-9.0.0/work/apache-arrow-9.0.0/cpp/thirdparty/hadoop/include/hdfs.h
-- All bundled static libraries:
-- CMAKE_C_FLAGS: -march=native -O2 -pipe -DNDEBUG
-- CMAKE_CXX_FLAGS: -march=native -O2 -pipe -DNDEBUG
-- Looking for backtrace
-- Looking for backtrace - found
-- backtrace facility detected in default set of libraries
-- Found Backtrace: /usr/include
-- Found Python3: /var/tmp/portage/dev-libs/apache-arrow-9.0.0/temp/python3.10/bin/python3 (found suitable version "3.10.8", minimum required is "3.7") found components: Interpreter Development.Module NumPy
-- Found Python3Alt: /var/tmp/portage/dev-libs/apache-arrow-9.0.0/temp/python3.10/bin/python3 (Required is at least version "3.7")
-- ---------------------------------------------------------------------
-- Arrow version: 9.0.0
--
-- Build configuration summary:
-- Generator: Ninja
-- Build type: RelWithDebInfo
-- Source directory: /var/tmp/portage/dev-libs/apache-arrow-9.0.0/work/apache-arrow-9.0.0/cpp
-- Install prefix: /usr
--
-- Compile and link options:
--
-- ARROW_CXXFLAGS=-DNDEBUG [default=""]
-- Compiler flags to append when compiling Arrow
-- ARROW_BUILD_STATIC=OFF [default=ON]
-- Build static libraries
-- ARROW_BUILD_SHARED=ON [default=ON]
-- Build shared libraries
-- ARROW_PACKAGE_KIND="" [default=""]
-- Arbitrary string that identifies the kind of package
-- (for informational purposes)
-- ARROW_GIT_ID="" [default=""]
-- The Arrow git commit id (if any)
-- ARROW_GIT_DESCRIPTION="" [default=""]
-- The Arrow git commit description (if any)
-- ARROW_NO_DEPRECATED_API=OFF [default=OFF]
-- Exclude deprecated APIs from build
-- ARROW_POSITION_INDEPENDENT_CODE=ON [default=ON]
-- Whether to create position-independent target
-- ARROW_USE_CCACHE=ON [default=ON]
-- Use ccache when compiling (if available)
-- ARROW_USE_LD_GOLD=OFF [default=OFF]
-- Use ld.gold for linking on Linux (if available)
-- ARROW_USE_PRECOMPILED_HEADERS=OFF [default=OFF]
-- Use precompiled headers when compiling
-- ARROW_SIMD_LEVEL=DEFAULT [default=NONE|SSE4_2|AVX2|AVX512|NEON|DEFAULT]
-- Compile-time SIMD optimization level
-- ARROW_RUNTIME_SIMD_LEVEL=MAX [default=NONE|SSE4_2|AVX2|AVX512|MAX]
-- Max runtime SIMD optimization level
-- ARROW_ARMV8_ARCH=armv8-a [default=armv8-a|armv8-a+crc+crypto]
-- Arm64 arch and extensions
-- ARROW_ALTIVEC=ON [default=ON]
-- Build with Altivec if compiler has support
-- ARROW_RPATH_ORIGIN=OFF [default=OFF]
-- Build Arrow libraries with RATH set to $ORIGIN
-- ARROW_INSTALL_NAME_RPATH=ON [default=ON]
-- Build Arrow libraries with install_name set to @rpath
-- ARROW_GGDB_DEBUG=ON [default=ON]
-- Pass -ggdb flag to debug builds
--
-- Test and benchmark options:
--
-- ARROW_BUILD_EXAMPLES=OFF [default=OFF]
-- Build the Arrow examples
-- ARROW_BUILD_TESTS=OFF [default=OFF]
-- Build the Arrow googletest unit tests
-- ARROW_ENABLE_TIMING_TESTS=ON [default=ON]
-- Enable timing-sensitive tests
-- ARROW_BUILD_INTEGRATION=OFF [default=OFF]
-- Build the Arrow integration test executables
-- ARROW_BUILD_BENCHMARKS=OFF [default=OFF]
-- Build the Arrow micro benchmarks
-- ARROW_BUILD_BENCHMARKS_REFERENCE=OFF [default=OFF]
-- Build the Arrow micro reference benchmarks
-- ARROW_BUILD_OPENMP_BENCHMARKS=OFF [default=OFF]
-- Build the Arrow benchmarks that rely on OpenMP
-- ARROW_BUILD_DETAILED_BENCHMARKS=OFF [default=OFF]
-- Build benchmarks that do a longer exploration of performance
-- ARROW_TEST_LINKAGE=shared [default=shared|static]
-- Linkage of Arrow libraries with unit tests executables.
-- ARROW_FUZZING=OFF [default=OFF]
-- Build Arrow Fuzzing executables
-- ARROW_LARGE_MEMORY_TESTS=OFF [default=OFF]
-- Enable unit tests which use large memory
--
-- Lint options:
--
-- ARROW_ONLY_LINT=OFF [default=OFF]
-- Only define the lint and check-format targets
-- ARROW_VERBOSE_LINT=OFF [default=OFF]
-- If off, 'quiet' flags will be passed to linting tools
-- ARROW_GENERATE_COVERAGE=OFF [default=OFF]
-- Build with C++ code coverage enabled
--
-- Checks options:
--
-- ARROW_TEST_MEMCHECK=OFF [default=OFF]
-- Run the test suite using valgrind --tool=memcheck
-- ARROW_USE_ASAN=OFF [default=OFF]
-- Enable Address Sanitizer checks
-- ARROW_USE_TSAN=OFF [default=OFF]
-- Enable Thread Sanitizer checks
-- ARROW_USE_UBSAN=OFF [default=OFF]
-- Enable Undefined Behavior sanitizer checks
--
-- Project component options:
--
-- ARROW_BUILD_UTILITIES=OFF [default=OFF]
-- Build Arrow commandline utilities
-- ARROW_COMPUTE=ON [default=OFF]
-- Build the Arrow Compute Modules
-- ARROW_CSV=ON [default=OFF]
-- Build the Arrow CSV Parser Module
-- ARROW_CUDA=OFF [default=OFF]
-- Build the Arrow CUDA extensions (requires CUDA toolkit)
-- ARROW_DATASET=ON [default=OFF]
-- Build the Arrow Dataset Modules
-- ARROW_SUBSTRAIT=OFF [default=OFF]
-- Build the Arrow Substrait Consumer Module
-- ARROW_FILESYSTEM=ON [default=OFF]
-- Build the Arrow Filesystem Layer
-- ARROW_FLIGHT=OFF [default=OFF]
-- Build the Arrow Flight RPC System (requires GRPC, Protocol Buffers)
-- ARROW_FLIGHT_SQL=OFF [default=OFF]
-- Build the Arrow Flight SQL extension
-- ARROW_GANDIVA=OFF [default=OFF]
-- Build the Gandiva libraries
-- ARROW_GCS=OFF [default=OFF]
-- Build Arrow with GCS support (requires the GCloud SDK for C++)
-- ARROW_HDFS=ON [default=OFF]
-- Build the Arrow HDFS bridge
-- ARROW_IPC=ON [default=ON]
-- Build the Arrow IPC extensions
-- ARROW_JEMALLOC=OFF [default=ON]
-- Build the Arrow jemalloc-based allocator
-- ARROW_JNI=OFF [default=OFF]
-- Build the Arrow JNI lib
-- ARROW_JSON=ON [default=OFF]
-- Build Arrow with JSON support (requires RapidJSON)
-- ARROW_MIMALLOC=OFF [default=OFF]
-- Build the Arrow mimalloc-based allocator
-- ARROW_PARQUET=ON [default=OFF]
-- Build the Parquet libraries
-- ARROW_ORC=OFF [default=OFF]
-- Build the Arrow ORC adapter
-- ARROW_PLASMA=OFF [default=OFF]
-- Build the plasma object store along with Arrow
-- ARROW_PLASMA_JAVA_CLIENT=OFF [default=OFF]
-- Build the plasma object store java client
-- ARROW_PYTHON=ON [default=OFF]
-- Build the Arrow CPython extensions
-- ARROW_S3=OFF [default=OFF]
-- Build Arrow with S3 support (requires the AWS SDK for C++)
-- ARROW_SKYHOOK=OFF [default=OFF]
-- Build the Skyhook libraries
-- ARROW_TENSORFLOW=OFF [default=OFF]
-- Build Arrow with TensorFlow support enabled
-- ARROW_TESTING=OFF [default=OFF]
-- Build the Arrow testing libraries
--
-- Thirdparty toolchain options:
--
-- ARROW_DEPENDENCY_SOURCE=SYSTEM [default=AUTO|BUNDLED|SYSTEM|CONDA|VCPKG|BREW]
-- Method to use for acquiring arrow's build dependencies
-- ARROW_VERBOSE_THIRDPARTY_BUILD=OFF [default=OFF]
-- Show output from ExternalProjects rather than just logging to files
-- ARROW_DEPENDENCY_USE_SHARED=ON [default=ON]
-- Link to shared libraries
-- ARROW_BOOST_USE_SHARED=ON [default=ON]
-- Rely on Boost shared libraries where relevant
-- ARROW_BROTLI_USE_SHARED=ON [default=ON]
-- Rely on Brotli shared libraries where relevant
-- ARROW_BZ2_USE_SHARED=ON [default=ON]
-- Rely on Bz2 shared libraries where relevant
-- ARROW_GFLAGS_USE_SHARED=ON [default=ON]
-- Rely on GFlags shared libraries where relevant
-- ARROW_GRPC_USE_SHARED=ON [default=ON]
-- Rely on gRPC shared libraries where relevant
-- ARROW_JEMALLOC_USE_SHARED=ON [default=ON]
-- Rely on jemalloc shared libraries where relevant
-- ARROW_LZ4_USE_SHARED=ON [default=ON]
-- Rely on lz4 shared libraries where relevant
-- ARROW_OPENSSL_USE_SHARED=ON [default=ON]
-- Rely on OpenSSL shared libraries where relevant
-- ARROW_PROTOBUF_USE_SHARED=ON [default=ON]
-- Rely on Protocol Buffers shared libraries where relevant
-- ARROW_SNAPPY_USE_SHARED=ON [default=ON]
-- Rely on snappy shared libraries where relevant
-- ARROW_THRIFT_USE_SHARED=ON [default=ON]
-- Rely on thrift shared libraries where relevant
-- ARROW_UTF8PROC_USE_SHARED=ON [default=ON]
-- Rely on utf8proc shared libraries where relevant
-- ARROW_ZSTD_USE_SHARED=ON [default=ON]
-- Rely on zstd shared libraries where relevant
-- ARROW_USE_GLOG=OFF [default=OFF]
-- Build libraries with glog support for pluggable logging
-- ARROW_WITH_BACKTRACE=ON [default=ON]
-- Build with backtrace support
-- ARROW_WITH_OPENTELEMETRY=OFF [default=OFF]
-- Build libraries with OpenTelemetry support for distributed tracing
-- ARROW_WITH_BROTLI=OFF [default=OFF]
-- Build with Brotli compression
-- ARROW_WITH_BZ2=ON [default=OFF]
-- Build with BZ2 compression
-- ARROW_WITH_LZ4=ON [default=OFF]
-- Build with lz4 compression
-- ARROW_WITH_SNAPPY=OFF [default=OFF]
-- Build with Snappy compression
-- ARROW_WITH_ZLIB=ON [default=OFF]
-- Build with zlib compression
-- ARROW_WITH_ZSTD=ON [default=OFF]
-- Build with zstd compression
-- ARROW_WITH_UCX=OFF [default=OFF]
-- Build with UCX transport for Arrow Flight
-- (only used if ARROW_FLIGHT is ON)
-- ARROW_WITH_UTF8PROC=ON [default=ON]
-- Build with support for Unicode properties using the utf8proc library
-- (only used if ARROW_COMPUTE is ON or ARROW_GANDIVA is ON)
-- ARROW_WITH_RE2=ON [default=ON]
-- Build with support for regular expressions using the re2 library
-- (only used if ARROW_COMPUTE or ARROW_GANDIVA is ON)
--
-- Parquet options:
--
-- PARQUET_MINIMAL_DEPENDENCY=OFF [default=OFF]
-- Depend only on Thirdparty headers to build libparquet.
-- Always OFF if building binaries
-- PARQUET_BUILD_EXECUTABLES=OFF [default=OFF]
-- Build the Parquet executable CLI tools. Requires static libraries to be built.
-- PARQUET_BUILD_EXAMPLES=OFF [default=OFF]
-- Build the Parquet examples. Requires static libraries to be built.
-- PARQUET_REQUIRE_ENCRYPTION=OFF [default=OFF]
-- Build support for encryption. Fail if OpenSSL is not found
--
-- Gandiva options:
--
-- ARROW_GANDIVA_JAVA=OFF [default=OFF]
-- Build the Gandiva JNI wrappers
-- ARROW_GANDIVA_STATIC_LIBSTDCPP=OFF [default=OFF]
-- Include -static-libstdc++ -static-libgcc when linking with
-- Gandiva static libraries
-- ARROW_GANDIVA_PC_CXX_FLAGS="" [default=""]
-- Compiler flags to append when pre-compiling Gandiva operations
--
-- Advanced developer options:
--
-- ARROW_EXTRA_ERROR_CONTEXT=OFF [default=OFF]
-- Compile with extra error context (line numbers, code)
-- ARROW_OPTIONAL_INSTALL=OFF [default=OFF]
-- If enabled install ONLY targets that have already been built. Please be
-- advised that if this is enabled 'install' will fail silently on components
-- that have not been built
-- Outputting build configuration summary to /var/tmp/portage/dev-libs/apache-arrow-9.0.0/work/apache-arrow-9.0.0/cpp_build/cmake_summary.json
-- <<< Gentoo configuration >>>
Build type RelWithDebInfo
Install path /usr
Compiler flags:
C -march=native -O2 -pipe -DNDEBUG
C++ -march=native -O2 -pipe -DNDEBUG
Linker flags:
Executable -Wl,-O1 -Wl,--as-needed
Module -Wl,-O1 -Wl,--as-needed
Shared -Wl,-O1 -Wl,--as-needed
-- Configuring done
CMake Error at CMakeLists.txt:862 (add_dependencies):
The dependency target "lz4::lz4" of target "arrow_dependencies" does not
exist.
CMake Error at cmake_modules/BuildUtils.cmake:250 (target_link_libraries):
Target "arrow_objlib" links to:
lz4::lz4
but the target was not found. Possible reasons include:
* There is a typo in the target name.
* A find_package call is missing for an IMPORTED target.
* An ALIAS target is missing.
Call Stack (most recent call first):
src/arrow/CMakeLists.txt:563 (add_arrow_lib)
CMake Error at cmake_modules/BuildUtils.cmake:311 (target_link_libraries):
Target "arrow_shared" links to:
lz4::lz4
but the target was not found. Possible reasons include:
* There is a typo in the target name.
* A find_package call is missing for an IMPORTED target.
* An ALIAS target is missing.
Call Stack (most recent call first):
src/arrow/CMakeLists.txt:563 (add_arrow_lib)
CMake Error at cmake_modules/BuildUtils.cmake:250 (target_link_libraries):
Target "arrow_dataset_objlib" links to:
lz4::lz4
but the target was not found. Possible reasons include:
* There is a typo in the target name.
* A find_package call is missing for an IMPORTED target.
* An ALIAS target is missing.
Call Stack (most recent call first):
src/arrow/dataset/CMakeLists.txt:51 (add_arrow_lib)
-- Generating done
CMake Generate step failed. Build files cannot be regenerated correctly.
* ERROR: dev-libs/apache-arrow-9.0.0-r1::spark-overlay failed (configure phase):
* cmake failed
*
* Call stack:
* ebuild.sh, line 122: Called src_configure
* environment, line 3267: Called cmake_src_configure
* environment, line 1717: Called die
* The specific snippet of code:
* "${CMAKE_BINARY}" "${cmakeargs[@]}" "${CMAKE_USE_DIR}" || die "cmake failed";I can confirm that disabling the -lz4 USE flag resolves this issue. Apache Arrow successfully builds without issue then. 👍 ⬆️
I'm guessing no one has sufficient time to debug this. Right? I don't, sadly. You're probably the same as me. If so, we should consider temporarily removing LZ4 support entirely by just:
- Dropping the
lz4USE flag. - Forcing
-DARROW_WITH_LZ4=OFFinsrc_configure().
Sad cat is sad. 😿
And While We're on the Subject...
It'd be great if we could also bump both the EAPI and PYTHON_COMPAT for Apache Arrow. The currently listed PYTHON_COMPAT is two versions behind, which also makes sad cat cry. Something resembling the following would be noice:
EAPI=8
PYTHON_COMPAT=( python3_{8..11} )I can confirm that that behaves as expected. That said, I see that @Leo3418 already bumped everything for the apache-arrow-4.0.1-r1 ebuild. That's great! But those changes didn't make it over to the apache-arrow-9.0.0-r1 ebuild, inexplicably. That's sad.
9.0.0-r1 Dependencies: They're Just Bad, Everybody
Moreover, it looks like the apache-arrow-4.0.1-r1 ebuild has a number of other dramatic improvements over the apache-arrow-9.0.0-r1 ebuild. The dependency list in the former is much safer and more complete than the dependency list in the latter.
This is what apache-arrow-4.0.1-r1 look like:
RDEPEND="
dev-java/brotli-dec
bzip2? ( app-arch/bzip2 )
lz4? ( app-arch/lz4:= )
parquet? (
dev-libs/libutf8proc:=
dev-libs/re2:=
dev-libs/thrift
)
python? (
${PYTHON_DEPS}
$(python_gen_cond_dep '
dev-python/numpy[${PYTHON_USEDEP}]
')
)
zlib? ( sys-libs/zlib )
zstd? ( app-arch/zstd:= )
"
DEPEND="
${RDEPEND}
dev-cpp/xsimd
dev-libs/rapidjson
net-libs/grpc
"That's good. Admittedly, the lack of BDEPEND is a bit disconcerting. But at least everything is conditional on USE flags and (more importantly) constrained with correct := slotting to implicitly recompile Apache Arrow when its underlying dependencies are recompiled.
On the other hand, this is what apache-arrow-9.0.0-r1 looks like:
RDEPEND="
python? (
${PYTHON_DEPS}
$(python_gen_cond_dep '
dev-python/numpy[${PYTHON_USEDEP}]
')
)
"
DEPEND="${RDEPEND}"
BDEPEND="
app-arch/lz4
>=dev-cpp/xsimd-8.1
dev-java/brotli-dec
dev-libs/libutf8proc
dev-libs/rapidjson
dev-libs/re2
dev-libs/thrift
net-libs/grpc
"That's horrible! 😱
Nothing is conditional on USE flags and nothing is constrained with correct := slotting. To compound matters, that BDEPEND looks super suspicious. I'm fairly certain that most or all of those supposedly BDEPEND dependencies are actually just DEPEND dependencies. Does anyone have any idea how these ebuilds became so desynchronized? It's... not great, guys.
Relatedly, I've been trying to track down an issue with Streamlit (which requires PyArrow, which requires Apache Arrow) for the past several days. Python's raising a non-human-readable Illegal instruction core dump at a certain point in our Streamlit app's lifecycle. I'm increasingly convinced that the apache-arrow-9.0.0-r1 ebuild is the underlying cause. Why? Because it fails to declare := slots, my Apache Arrow installation silently broke when underlying dependencies were recompiled after my most recent emerge @world.
This is why sad cat no longer smiles.
Let Me Fix Everything for Everyone
I desperately need Apache Arrow to just work. I'll happily submit a PR in a day or two fixing everything above for everyone, including:
- Temporarily disabling the
lz4USE flag. - Bumping 9.0.0
EAPIandPYTHON_COMPAT. - Fixing 9.0.0 dependencies.
Thanks for all the tremendous hard work! You're the spark that inspires me to keep using Java.