Skip to content

Commit

Permalink
Remove the getStageInfo() method from SparkContext.
Browse files Browse the repository at this point in the history
This method exposes the Stage objects, which are
private to Spark and should not be exposed to the
user.

This method was added in apache@01d77f3; ccing @squito here in case there's a good reason to keep this!

Author: Kay Ousterhout <[email protected]>

Closes apache#308 from kayousterhout/remove_public_method and squashes the following commits:

2e2f009 [Kay Ousterhout] Remove the getStageInfo() method from SparkContext.
  • Loading branch information
kayousterhout authored and mateiz committed Apr 5, 2014
1 parent 7c18428 commit 2d0150c
Show file tree
Hide file tree
Showing 2 changed files with 1 addition and 5 deletions.
4 changes: 0 additions & 4 deletions core/src/main/scala/org/apache/spark/SparkContext.scala
Original file line number Diff line number Diff line change
Expand Up @@ -731,10 +731,6 @@ class SparkContext(
*/
def getPersistentRDDs: Map[Int, RDD[_]] = persistentRdds.toMap

def getStageInfo: Map[Stage, StageInfo] = {
dagScheduler.stageToInfos
}

/**
* Return information about blocks stored in all of the slaves
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ class DAGScheduler(
private[scheduler] val shuffleToMapStage = new TimeStampedHashMap[Int, Stage]
private[scheduler] val jobIdToActiveJob = new HashMap[Int, ActiveJob]
private[scheduler] val resultStageToJob = new HashMap[Stage, ActiveJob]
private[spark] val stageToInfos = new TimeStampedHashMap[Stage, StageInfo]
private[scheduler] val stageToInfos = new TimeStampedHashMap[Stage, StageInfo]

// Stages we need to run whose parents aren't done
private[scheduler] val waitingStages = new HashSet[Stage]
Expand Down

0 comments on commit 2d0150c

Please sign in to comment.