[SPARK-2284][UI] Mark all failed tasks as failures. #1224
Conversation
Previously only tasks failed with ExceptionFailure reason was marked as failure.
|
@pwendell If still possible, this should go into 1.0.1. |
| (None, Option(taskEnd.taskMetrics)) | ||
| case e: org.apache.spark.TaskEndReason => | ||
| stageIdToTasksFailed(sid) = stageIdToTasksFailed.getOrElse(sid, 0) + 1 | ||
| (None, None) |
There was a problem hiding this comment.
I will submit a PR later to consolidate the error reporting (right now we have some pretty convoluted error reporting in the UI).
|
Merged build triggered. |
|
Merged build started. |
|
Merged build finished. All automated tests passed. |
|
All automated tests passed. |
|
LGTM, feel free to merge. |
| @@ -185,12 +185,15 @@ class JobProgressListener(conf: SparkConf) extends SparkListener { | |||
|
|
|||
| val (failureInfo, metrics): (Option[ExceptionFailure], Option[TaskMetrics]) = | |||
| taskEnd.reason match { | |||
| case org.apache.spark.Success => | |||
There was a problem hiding this comment.
why'd you make these fully qualified instead of importing the class? is there a naming conflict?
There was a problem hiding this comment.
Success is too common ....
There was a problem hiding this comment.
|
Ok I'm going to merge this in master & branch-1.0. Thanks for taking a look. |
Previously only tasks failed with ExceptionFailure reason was marked as failure. Author: Reynold Xin <[email protected]> Closes #1224 from rxin/SPARK-2284 and squashes the following commits: be79dbd [Reynold Xin] [SPARK-2284][UI] Mark all failed tasks as failures. (cherry picked from commit 4a346e2) Signed-off-by: Reynold Xin <[email protected]>
Previously only tasks failed with ExceptionFailure reason was marked as failure. Author: Reynold Xin <[email protected]> Closes apache#1224 from rxin/SPARK-2284 and squashes the following commits: be79dbd [Reynold Xin] [SPARK-2284][UI] Mark all failed tasks as failures.
…EWED_PARTITION_THRESHOLD in HandleOuterJoinBuildSideSkew (#1224) * [CARMEL-6542] Change ADVISORY_PARTITION_SIZE_IN_BYTES to SKEW_JOIN_SKEWED_PARTITION_THRESHOLD in HandleOuterJoinBuildSideSkew * [CARMEL-6542] Change ADVISORY_PARTITION_SIZE_IN_BYTES to SKEW_JOIN_SKEWED_PARTITION_THRESHOLD in HandleOuterJoinBuildSideSkew * Update HandleOuterJoinBuildSideSkewSuite.scala
Previously only tasks failed with ExceptionFailure reason was marked as failure.