[VL] Q97 cross-task spilling hangs

[zhztheplayer]

Happens when Velox task A grows buffers created by task B, during which spill is requested on task A again.

Stack:

syscall 0x00007f53533e673d
folly::detail::nativeFutexWaitImpl Futex.cpp:126
folly::detail::futexWaitImpl Futex.cpp:254
folly::detail::futexWait<…> Futex-inl.h:94
folly::detail::MemoryIdler::futexWait<…> MemoryIdler.h:128
folly::fibers::Baton::waitThread Baton.cpp:70
folly::fibers::Baton::wait<…> Baton-inl.h:54
folly::fibers::Baton::wait Baton.cpp:46
folly::futures::detail::waitImpl<…> Future-inl.h:2230
folly::SemiFuture<folly::Unit>::wait() & Future-inl.h:2337
folly::SemiFuture<folly::Unit>::wait() && Future-inl.h:2343
facebook::velox::exec::Task::MemoryReclaimer::reclaimTask Task.cpp:2933
facebook::velox::exec::Task::MemoryReclaimer::reclaim Task.cpp:2907
facebook::velox::memory::MemoryReclaimer::reclaim MemoryArbitrator.cpp:266
gluten::ListenableArbitrator::shrinkCapacity VeloxMemoryManager.cc:122
gluten::WholeStageResultIterator::spillFixedSize WholeStageResultIterator.cc:239
gluten::ResultIterator::spillFixedSize ResultIterator.h:71
Java_org_apache_gluten_vectorized_ColumnarBatchOutIterator_nativeSpill JniWrapper.cc:516
<unknown> 0x00007f533d018427
<unknown> 0x0000000000800000
<unknown> 0x00007f4e38003260
<unknown> 0x00000007c1420c28
<unknown> 0x00007f52d8dc7890
<unknown> 0x00007f4e38003000
<unknown> 0x00007f52d8dc7838
<unknown> 0x0000000000000000

Log:

https://github.com/apache/incubator-gluten/actions/runs/10856089295/job/30131530452?pr=7237