Skip to content

Commit

Permalink
[SPARK-48289][DOCKER][TEST] Clean up Oracle JDBC tests by skipping re…
Browse files Browse the repository at this point in the history
…dundant SYSTEM password reset

### What changes were proposed in this pull request?
This pull request improves the Oracle JDBC tests by skipping the redundant SYSTEM password reset.

### Why are the changes needed?
These changes are necessary to clean up the Oracle JDBC tests.

This pull request effectively reverts the modifications introduced in [SPARK-46592](https://issues.apache.org/jira/browse/SPARK-46592) and [PR #44594](#44594), which attempted to work around the sporadic occurrence of ORA-65048 and ORA-04021 errors by setting the Oracle parameter DDL_LOCK_TIMEOUT.

As discussed in [issue #35](gvenzl/oci-oracle-free#35), setting DDL_LOCK_TIMEOUT did not resolve the issue. The root cause appears to be an Oracle bug or unwanted behavior related to the use of Pluggable Database (PDB) rather than the expected functionality of Oracle itself.

Additionally, with [SPARK-48141](https://issues.apache.org/jira/browse/SPARK-48141), we have upgraded the Oracle version used in the tests to Oracle Free 23ai, version 23.4. This upgrade should help address some of the issues observed with the previous Oracle version.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
This patch was tested using the existing test suite, with a particular focus on Oracle JDBC tests. The following steps were executed:
```
export ENABLE_DOCKER_INTEGRATION_TESTS=1
./build/sbt -Pdocker-integration-tests "docker-integration-tests/testOnly org.apache.spark.sql.jdbc.OracleIntegrationSuite"
```

### Was this patch authored or co-authored using generative AI tooling?
No

Closes #46598 from LucaCanali/fixOracleIntegrationTests.

Lead-authored-by: Kent Yao <[email protected]>
Co-authored-by: Luca Canali <[email protected]>
Signed-off-by: Kent Yao <[email protected]>
  • Loading branch information
yaooqinn and LucaCanali committed May 16, 2024
1 parent ca35932 commit 4ffaa2e
Show file tree
Hide file tree
Showing 4 changed files with 13 additions and 44 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,7 @@

package org.apache.spark.sql.jdbc

import java.io.{File, PrintWriter}

import com.github.dockerjava.api.model._

import org.apache.spark.internal.Logging
import org.apache.spark.util.Utils

class OracleDatabaseOnDocker extends DatabaseOnDocker with Logging {
lazy override val imageName =
Expand All @@ -38,30 +33,4 @@ class OracleDatabaseOnDocker extends DatabaseOnDocker with Logging {
override def getJdbcUrl(ip: String, port: Int): String = {
s"jdbc:oracle:thin:system/$oracle_password@//$ip:$port/freepdb1"
}

override def beforeContainerStart(
hostConfigBuilder: HostConfig,
containerConfigBuilder: ContainerConfig): Unit = {
try {
val dir = Utils.createTempDir()
val writer = new PrintWriter(new File(dir, "install.sql"))
// SPARK-46592: gvenzl/oracle-free occasionally fails to start with the following error:
// 'ORA-04021: timeout occurred while waiting to lock object', when initializing the
// SYSTEM user. This is due to the fact that the default DDL_LOCK_TIMEOUT is 0, which
// means that the lock will no wait. We set the timeout to 30 seconds to try again.
// TODO: This workaround should be removed once the issue is fixed in the image.
// https://github.com/gvenzl/oci-oracle-free/issues/35
writer.write("ALTER SESSION SET DDL_LOCK_TIMEOUT = 30;\n")
writer.write(s"""ALTER USER SYSTEM IDENTIFIED BY "$oracle_password";""")
writer.close()
val newBind = new Bind(
dir.getAbsolutePath,
new Volume("/docker-entrypoint-initdb.d"),
AccessMode.DEFAULT)
hostConfigBuilder.withBinds(hostConfigBuilder.getBinds :+ newBind: _*)
} catch {
case e: Exception =>
logWarning("Failed to create install.sql file", e)
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -54,11 +54,11 @@ import org.apache.spark.tags.DockerTest
* A sequence of commands to build the Oracle Database Free container image:
* $ git clone https://github.com/oracle/docker-images.git
* $ cd docker-images/OracleDatabase/SingleInstance/dockerfiles
* $ ./buildContainerImage.sh -v 23.2.0 -f
* $ export ORACLE_DOCKER_IMAGE_NAME=oracle/database:23.2.0-free
* $ ./buildContainerImage.sh -v 23.4.0 -f
* $ export ORACLE_DOCKER_IMAGE_NAME=oracle/database:23.4.0-free
*
* This procedure has been validated with Oracle Database Free version 23.2.0,
* and with Oracle Express Edition versions 18.4.0 and 21.3.0
* This procedure has been validated with Oracle Database Free version 23.4.0,
* and with Oracle Express Edition versions 18.4.0 and 21.4.0
*/
@DockerTest
class OracleIntegrationSuite extends DockerJDBCIntegrationSuite with SharedSparkSession {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,12 +48,12 @@ import org.apache.spark.tags.DockerTest
*
* A sequence of commands to build the Oracle Database Free container image:
* $ git clone https://github.com/oracle/docker-images.git
* $ cd docker-images/OracleDatabase/SingleInstance/dockerfiles
* $ ./buildContainerImage.sh -v 23.2.0 -f
* $ export ORACLE_DOCKER_IMAGE_NAME=oracle/database:23.2.0-free
* $ cd docker-images/OracleDatabase/SingleInstance/dockerfiles0
* $ ./buildContainerImage.sh -v 23.4.0 -f
* $ export ORACLE_DOCKER_IMAGE_NAME=oracle/database:23.4.0-free
*
* This procedure has been validated with Oracle Database Free version 23.2.0,
* and with Oracle Express Edition versions 18.4.0 and 21.3.0
* This procedure has been validated with Oracle Database Free version 23.4.0,
* and with Oracle Express Edition versions 18.4.0 and 21.4.0
*/
@DockerTest
class OracleIntegrationSuite extends DockerJDBCIntegrationV2Suite with V2JDBCTest {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -46,11 +46,11 @@ import org.apache.spark.tags.DockerTest
* A sequence of commands to build the Oracle Database Free container image:
* $ git clone https://github.com/oracle/docker-images.git
* $ cd docker-images/OracleDatabase/SingleInstance/dockerfiles
* $ ./buildContainerImage.sh -v 23.2.0 -f
* $ export ORACLE_DOCKER_IMAGE_NAME=oracle/database:23.2.0-free
* $ ./buildContainerImage.sh -v 23.4.0 -f
* $ export ORACLE_DOCKER_IMAGE_NAME=oracle/database:23.4.0-free
*
* This procedure has been validated with Oracle Database Free version 23.2.0,
* and with Oracle Express Edition versions 18.4.0 and 21.3.0
* This procedure has been validated with Oracle Database Free version 23.4.0,
* and with Oracle Express Edition versions 18.4.0 and 21.4.0
*/
@DockerTest
class OracleNamespaceSuite extends DockerJDBCIntegrationSuite with V2JDBCNamespaceTest {
Expand Down

0 comments on commit 4ffaa2e

Please sign in to comment.