[SPARK-26352][SQL][FOLLOWUP-2.3] Fix missing sameOutput in branch-2.3

This is the branch-2.3 equivalent of apache#23330. After apache#23303 was merged to branch-2.3/2.4, the builds on those branches were broken due to missing a `LogicalPlan.sameOutput` function which came from apache#22713 only available on master. This PR is to follow-up with the broken 2.3/2.4 branches and make a copy of the new `LogicalPlan.sameOutput` into `ReorderJoin` to make it locally available. Fix the build of 2.3/2.4. Closes apache#23333 from rednaxelafx/branch-2.3. Authored-by: Kris Mok <[email protected]> Signed-off-by: Wenchen Fan <[email protected]>
TIBCOSoftware · Aug 14, 2021 · 7f38eb0 · 7f38eb0
1 parent 9c45cb9
commit 7f38eb0
Showing 1 changed file with 16 additions and 1 deletion.
diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
@@ -87,14 +87,29 @@ object ReorderJoin extends Rule[LogicalPlan] with PredicateHelper {
     case p @ ExtractFiltersAndInnerJoins(input, conditions)
         if input.size > 2 && conditions.nonEmpty =>
       val reordered = createOrderedJoin(input, conditions)
-      if (p.sameOutput(reordered)) {
+      if (sameOutput(p, reordered)) {
         reordered
       } else {
         // Reordering the joins have changed the order of the columns.
         // Inject a projection to make sure we restore to the expected ordering.
         Project(p.output, reordered)
       }
   }
+
+  /**
+   * Returns true iff output of both plans are semantically the same, ie.:
+   *  - they contain the same number of `Attribute`s;
+   *  - references are the same;
+   *  - the order is equal too.
+   * NOTE: this is copied over from SPARK-25691 from master.
+   */
+  def sameOutput(plan1: LogicalPlan, plan2: LogicalPlan): Boolean = {
+    val output1 = plan1.output
+    val output2 = plan2.output
+    output1.length == output2.length && output1.zip(output2).forall {
+      case (a1, a2) => a1.semanticEquals(a2)
+    }
+  }
 }
 
 /**