[SPARK-40382][SQL][TESTS][FOLLOW-UP] Make the tests pass with ANSI en…

…abled ### What changes were proposed in this pull request? This PR is a followup of #37825, that change the types in the test relation to make the tests pass with ANSI enalbed. ### Why are the changes needed? To recover the test coverage. Currently it fails with ANSI mode on: https://github.com/apache/spark/actions/runs/3246829492/jobs/5326051798#step:9:20487 ``` [info] - SPARK-40382: eliminate multiple distinct groups due to superficial differences *** FAILED *** (5 milliseconds) [info] org.apache.spark.sql.AnalysisException: [DATATYPE_MISMATCH.BINARY_OP_WRONG_TYPE] Cannot resolve "(b + c)" due to data type mismatch: the binary operator requires the input type ("NUMERIC" or "INTERVAL DAY TO SECOND" or "INTERVAL YEAR TO MONTH" or "INTERVAL"), not "STRING". [info] at org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.dataTypeMismatch(package.scala:68) [info] at org.apache.spark.sql.catalyst.analysis.CheckAnalysis.$anonfun$checkAnalysis0$5(CheckAnalysis.scala:223) [info] at org.apache.spark.sql.catalyst.analysis.CheckAnalysis.$anonfun$checkAnalysis0$5$adapted(CheckAnalysis.scala:210) [info] at org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:295) [info] at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreachUp$1(TreeNode.scala:294) [info] at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreachUp$1$adapted(TreeNode.scala:294) [info] at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62) [info] at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55) [info] at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49) [info] at org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:294) [info] at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreachUp$1(TreeNode.scala:2 ``` ### Does this PR introduce _any_ user-facing change? No, test-only. ### How was this patch tested? Manually ran the tests locally. Closes #38250 from HyukjinKwon/SPARK-40382-followup. Authored-by: Hyukjin Kwon <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]>
apache · Oct 14, 2022 · 8a28e91 · 8a28e91
1 parent 9d28909
commit 8a28e91
Showing 1 changed file with 3 additions and 2 deletions.
diff --git a/...c/test/scala/org/apache/spark/sql/catalyst/optimizer/RewriteDistinctAggregatesSuite.scala b/...c/test/scala/org/apache/spark/sql/catalyst/optimizer/RewriteDistinctAggregatesSuite.scala
@@ -28,6 +28,7 @@ class RewriteDistinctAggregatesSuite extends PlanTest {
   val nullInt = Literal(null, IntegerType)
   val nullString = Literal(null, StringType)
   val testRelation = LocalRelation($"a".string, $"b".string, $"c".string, $"d".string, $"e".int)
+  val testRelation2 = LocalRelation($"a".double, $"b".int, $"c".int, $"d".int, $"e".int)
 
   private def checkRewrite(rewrite: LogicalPlan): Unit = rewrite match {
     case Aggregate(_, _, Aggregate(_, _, _: Expand)) =>
@@ -77,7 +78,7 @@ class RewriteDistinctAggregatesSuite extends PlanTest {
   }
 
   test("SPARK-40382: eliminate multiple distinct groups due to superficial differences") {
-    val input = testRelation
+    val input = testRelation2
       .groupBy($"a")(
         countDistinct($"b" + $"c").as("agg1"),
         countDistinct($"c" + $"b").as("agg2"),
@@ -92,7 +93,7 @@ class RewriteDistinctAggregatesSuite extends PlanTest {
   }
 
   test("SPARK-40382: reduce multiple distinct groups due to superficial differences") {
-    val input = testRelation
+    val input = testRelation2
       .groupBy($"a")(
         countDistinct($"b" + $"c" + $"d").as("agg1"),
         countDistinct($"d" + $"c" + $"b").as("agg2"),