Skip to content

Commit

Permalink
[SPARK-3481] [SQL] Eliminate the error log in local Hive comparison test
Browse files Browse the repository at this point in the history
Logically, we should remove the Hive Table/Database first and then reset the Hive configuration, repoint to the new data warehouse directory etc.
Otherwise it raised exceptions like "Database doesn't not exists: default" in the local testing.

Author: Cheng Hao <hao.cheng@intel.com>

Closes #2352 from chenghao-intel/test_hive and squashes the following commits:

74fd76b [Cheng Hao] eliminate the error log

(cherry picked from commit 8194fc6)
Signed-off-by: Michael Armbrust <michael@databricks.com>
chenghao-intel authored and marmbrus committed Sep 12, 2014
1 parent ae6f554 commit 4f991c9
Showing 1 changed file with 8 additions and 9 deletions.
17 changes: 8 additions & 9 deletions sql/hive/src/main/scala/org/apache/spark/sql/hive/TestHive.scala
Original file line number Diff line number Diff line change
@@ -292,15 +292,6 @@ class TestHiveContext(sc: SparkContext) extends LocalHiveContext(sc) {
logger.asInstanceOf[org.apache.log4j.Logger].setLevel(org.apache.log4j.Level.WARN)
}

// It is important that we RESET first as broken hooks that might have been set could break
// other sql exec here.
runSqlHive("RESET")
// For some reason, RESET does not reset the following variables...
runSqlHive("set datanucleus.cache.collections=true")
runSqlHive("set datanucleus.cache.collections.lazy=true")
// Lots of tests fail if we do not change the partition whitelist from the default.
runSqlHive("set hive.metastore.partition.name.whitelist.pattern=.*")

loadedTables.clear()
catalog.client.getAllTables("default").foreach { t =>
logger.debug(s"Deleting table $t")
@@ -326,6 +317,14 @@ class TestHiveContext(sc: SparkContext) extends LocalHiveContext(sc) {
FunctionRegistry.unregisterTemporaryUDF(udfName)
}

// It is important that we RESET first as broken hooks that might have been set could break
// other sql exec here.
runSqlHive("RESET")
// For some reason, RESET does not reset the following variables...
runSqlHive("set datanucleus.cache.collections=true")
runSqlHive("set datanucleus.cache.collections.lazy=true")
// Lots of tests fail if we do not change the partition whitelist from the default.
runSqlHive("set hive.metastore.partition.name.whitelist.pattern=.*")
configure()

runSqlHive("USE default")

0 comments on commit 4f991c9

Please sign in to comment.