Closed
Description
Describe the bug
I just saw this when running benchmarks with latest from main and with xxhash64 disabled.
Caused by: org.apache.comet.CometNativeException: General execution error with reason org.apache.comet.CometNativeException: range end index 294912 out of range for slice of length 147456
at comet::errors::init::{{closure}}(__internal__:0)
at std::panicking::rust_panic_with_hook(__internal__:0)
at std::panicking::begin_panic_handler::{{closure}}(__internal__:0)
at std::sys_common::backtrace::__rust_end_short_backtrace(__internal__:0)
at rust_begin_unwind(__internal__:0)
at core::panicking::panic_fmt(__internal__:0)
at core::slice::index::slice_end_index_len_fail(__internal__:0)
at arrow_data::transform::variable_size::build_extend::{{closure}}(__internal__:0)
at arrow_data::transform::MutableArrayData::extend(__internal__:0)
at comet::execution::operators::copy_array(__internal__:0)
at comet::execution::operators::copy_or_cast_array(__internal__:0)
at <comet::execution::operators::copy::CopyStream as futures_core::stream::Stream>::poll_next(__internal__:0)
at <datafusion_physical_plan::stream::RecordBatchStreamAdapter<S> as futures_core::stream::Stream>::poll_next(__internal__:0)
at Java_org_apache_comet_Native_executePlan(__internal__:0)
at <unknown>(__internal__:0).
at org.apache.comet.Native.executePlan(Native Method)
at org.apache.comet.CometExecIterator.getNextBatch(CometExecIterator.scala:102)
at org.apache.comet.CometExecIterator.hasNext(CometExecIterator.scala:125)
at org.apache.spark.sql.comet.execution.shuffle.CometShuffleWriteProcessor.write(CometShuffleExchangeExec.scala:492)
Steps to reproduce
No response
Expected behavior
No response
Additional context
No response