Skip to content

Commit

Permalink
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
fallback on reused broadcast exchange
Browse files Browse the repository at this point in the history
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
zhouyuan committed Mar 20, 2022

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
1 parent cf38196 commit ddc8631
Showing 1 changed file with 9 additions and 2 deletions.
Original file line number Diff line number Diff line change
@@ -209,8 +209,15 @@ case class ColumnarPreOverrides() extends Rule[SparkPlan] {
right)
case plan: BroadcastQueryStageExec =>
logDebug(
s"Columnar Processing for ${plan.getClass} is currently supported, actual plan is ${plan.plan.getClass}.")
plan
s"Columnar Processing for ${plan.getClass} is currently supported, actual plan is ${plan.plan}.")
plan.plan match {
case ReusedExchangeExec(_, originalBroadcastPlan: ColumnarBroadcastExchangeAdaptor) =>
val newBroadcast = BroadcastExchangeExec(
originalBroadcastPlan.mode,
DataToArrowColumnarExec(plan.plan, 1))
SparkShimLoader.getSparkShims.newBroadcastQueryStageExec(plan.id, newBroadcast)
case other => plan
}
case plan: BroadcastExchangeExec =>
val child = replaceWithColumnarPlan(plan.child)
logDebug(s"Columnar Processing for ${plan.getClass} is currently supported.")

0 comments on commit ddc8631

Please sign in to comment.