Skip to content
This repository has been archived by the owner on Sep 18, 2023. It is now read-only.

columnar whole stage codegen failed due to empty results #1

Closed
zhouyuan opened this issue Dec 28, 2020 · 1 comment · Fixed by #36
Closed

columnar whole stage codegen failed due to empty results #1

zhouyuan opened this issue Dec 28, 2020 · 1 comment · Fixed by #36
Labels
bug Something isn't working

Comments

@zhouyuan
Copy link
Collaborator

We should guard the empty results code path

java.io.IOException: nativeEvaluate: evaluate failed with error msg Invalid: RecordBatch must be non-empty.
	at com.intel.oap.vectorized.ExpressionEvaluatorJniWrapper.nativeEvaluate(Native Method)
	at com.intel.oap.vectorized.ExpressionEvaluator.evaluate(ExpressionEvaluator.java:137)
	at com.intel.oap.vectorized.ExpressionEvaluator.evaluate(ExpressionEvaluator.java:107)
	at com.intel.oap.execution.ColumnarShuffledHashJoinExec.$anonfun$doExecuteColumnar$1(ColumnarShuffledHashJoinExec.scala:229)
	at org.apache.spark.rdd.ZippedPartitionsRDD2.compute(ZippedPartitionsRDD.scala:89)
@zhouyuan
Copy link
Collaborator Author

should check if the dep_cb is empty first
https://github.com/oap-project/native-sql-engine/blob/master/core/src/main/scala/com/intel/oap/execution/ColumnarWholeStageCodegenExec.scala#L347-L355

              (0 until dep_cb.numCols).toList.foreach(i =>
                dep_cb.column(i).asInstanceOf[ArrowWritableColumnVector].retain())
              buildRelationBatchHolder += dep_cb
              val beforeEval = System.nanoTime()
              val dep_rb = ConverterUtils.createArrowRecordBatch(dep_cb)
              hashRelationKernel.evaluate(dep_rb)
              ConverterUtils.releaseArrowRecordBatch(dep_rb)
              build_elapse += System.nanoTime() - beforeEval
              build_elapse_internal += System.nanoTime() - beforeEval

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant