Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-13747][SQL] Fix concurrent query with fork-join pool #11586

Closed
wants to merge 3 commits into from
Closed

[SPARK-13747][SQL] Fix concurrent query with fork-join pool #11586

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
661c28e
Fix concurrent query with fork-join pool
Mar 8, 2016
e1fd685
Merge branch 'master' of github.com:apache/spark into fix-concurrent-sql
Mar 9, 2016
9273874
Fix test
Mar 9, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -613,7 +613,12 @@ class DAGScheduler(
properties: Properties): Unit = {
val start = System.nanoTime
val waiter = submitJob(rdd, func, partitions, callSite, resultHandler, properties)
Await.ready(waiter.completionFuture, atMost = Duration.Inf)
// Note: Do not call Await.ready(future) because that calls `scala.concurrent.blocking`,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BlockManager#doPut() does this on line 891:

Await.ready(replicationFuture, Duration.Inf)

Should that be replaced as well ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No necessary. The issue here is because the ThreadLocal properties doesn't work well with ForkJoinPool. A new task may see some ThreadLocal properties from other task since they are using the same thread.

Does BlockManager use ThreadLocal?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently no.

// which causes concurrent SQL executions to fail if a fork-join pool is used. Note that
// due to idiosyncrasies in Scala, `awaitPermission` is not actually used anywhere so it's
// safe to pass in null here. For more detail, see SPARK-13747.
val awaitPermission = null.asInstanceOf[scala.concurrent.CanAwait]
waiter.completionFuture.ready(Duration.Inf)(awaitPermission)
waiter.completionFuture.value.get match {
case scala.util.Success(_) =>
logInfo("Job %d finished: %s, took %f s".format
Expand Down
12 changes: 12 additions & 0 deletions sql/core/src/test/scala/org/apache/spark/sql/execution/SQLExecutionSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,18 @@ class SQLExecutionSuite extends SparkFunSuite {
}
}

test("concurrent query execution with fork-join pool (SPARK-13747)") {
val sc = new SparkContext("local[*]", "test")
try {
// Should not throw IllegalArgumentException
(1 to 100).par.foreach { _ =>
sc.parallelize(1 to 5).map { i => (i, i) }.toDF("a", "b").count()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missed sqlcontext.implicits

}
} finally {
sc.stop()
}
}

/**
* Trigger SPARK-10548 by mocking a parent and its child thread executing queries concurrently.
*/
Expand Down