Skip to content

Commit

Permalink
[SPARK-32287][TESTS][FOLLOWUP] Add debugging for flaky ExecutorAlloca…
Browse files Browse the repository at this point in the history
…tionManagerSuite

### What changes were proposed in this pull request?

fixing flaky test in ExecutorAllocationManagerSuite. The issue is that there is a timing issue when we do a reset as to when the numExecutorsToAddPerResourceProfileId gets reset. The fix is to just always set those values back to 1 when we call reset().

### Why are the changes needed?

fixing flaky test in ExecutorAllocationManagerSuite

### Does this PR introduce _any_ user-facing change?

no
### How was this patch tested?

ran the unit test via this PR a bunch of times and the fix seems to be working.

Closes #29508 from tgravescs/debugExecAllocTest.

Authored-by: Thomas Graves <[email protected]>
Signed-off-by: HyukjinKwon <[email protected]>
  • Loading branch information
tgravescs authored and HyukjinKwon committed Aug 25, 2020
1 parent a3179a7 commit 2feab4e
Showing 1 changed file with 3 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -279,6 +279,9 @@ private[spark] class ExecutorAllocationManager(
numExecutorsTargetPerResourceProfileId.keys.foreach { rpId =>
numExecutorsTargetPerResourceProfileId(rpId) = initialNumExecutors
}
numExecutorsToAddPerResourceProfileId.keys.foreach { rpId =>
numExecutorsToAddPerResourceProfileId(rpId) = 1
}
executorMonitor.reset()
}

Expand Down

0 comments on commit 2feab4e

Please sign in to comment.