[VL] shuffle writer failed - Invalid: Failed to insert rows

[FelixYBW]

Backend

VL (Velox)

Bug description

24/08/13 13:28:42 ERROR [Executor task launch worker for task 6582.0 in stage 9.0 (TID 15705)] util.TaskResources: Task 15705 failed by error: 
org.apache.gluten.exception.GlutenException: Native write: shuffle writer failed - Invalid: Failed to insert rows. Remaining rows: 854
	at org.apache.gluten.vectorized.ShuffleWriterJniWrapper.write(Native Method)
	at org.apache.spark.shuffle.ColumnarShuffleWriter.internalWrite(ColumnarShuffleWriter.scala:180)
	at org.apache.spark.shuffle.ColumnarShuffleWriter.write(ColumnarShuffleWriter.scala:239)
	at org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:59)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:52)
	at org.apache.spark.scheduler.Task.run(Task.scala:131)
	at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:506)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1471)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:509)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)
E20240813 13:28:44.709362 161310 Exceptions.h:67] Line: /home/binweiyang/gluten/ep/build-velox/build/velox_ep/velox/exec/Task.cpp:1864, Function:terminate, Expression:  Cancelled, Source: RUNTIME, ErrorCode: INVALID_STATE
24/08/13 13:28:44 ERROR [Executor task launch worker for task 6582.0 in stage 9.0 (TID 15705)] executor.Executor: Exception in task 6582.0 in stage 9.0 (TID 15705)
org.apache.gluten.exception.GlutenException: Native write: shuffle writer failed - Invalid: Failed to insert rows. Remaining rows: 854
	at org.apache.gluten.vectorized.ShuffleWriterJniWrapper.write(Native Method)
	at org.apache.spark.shuffle.ColumnarShuffleWriter.internalWrite(ColumnarShuffleWriter.scala:180)
	at org.apache.spark.shuffle.ColumnarShuffleWriter.write(ColumnarShuffleWriter.scala:239)
	at org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:59)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:52)
	at org.apache.spark.scheduler.Task.run(Task.scala:131)
	at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:506)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1471)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:509)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)
24/08/13 13:37:36 ERROR [SIGTERM handler] executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM

Spark version

Spark-3.2.x

Spark configurations

No response

System information

No response

Relevant logs

No response

[FelixYBW]

fixed by 6832