SPARK-1689: Spark application should die when removed by Master

scheduler.error() will mask the error if there are active tasks. Being removed is a cataclysmic event for Spark applications, and should probably be treated as such. Author: Aaron Davidson <[email protected]> Closes apache#832 from aarondav/i-love-u and squashes the following commits: 9f1200f [Aaron Davidson] SPARK-1689: Spark application should die when removed by Master
pdeyhim · Jun 25, 2014 · ad2dcf3 · ad2dcf3
1 parent e712265
commit ad2dcf3
Showing 1 changed file with 2 additions and 0 deletions.
diff --git a/core/src/main/scala/org/apache/spark/scheduler/cluster/SparkDeploySchedulerBackend.scala b/core/src/main/scala/org/apache/spark/scheduler/cluster/SparkDeploySchedulerBackend.scala
@@ -88,6 +88,8 @@ private[spark] class SparkDeploySchedulerBackend(
     if (!stopping) {
       logError("Application has been killed. Reason: " + reason)
       scheduler.error(reason)
+      // Ensure the application terminates, as we can no longer run jobs.
+      sc.stop()
     }
   }