flux-hwloc: remove ignore of HWLOC_OBJ_GROUP
#3046
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The flux-sched build is failing because this PR is not based on current master. Ok to do an automated rebase via mergify? |
grondo
approved these changes
Jul 14, 2020
|
Ah, nevermind. I will set MWP. |
|
Thanks @grondo! |
|
@Mergifyio rebase |
Problem: When the `HWLOC_OBJ_GROUP` is ignored, the GPUs on the Sierra/Lassen clusters are represented in the resource topology as direct children of the node. This topology ignores the fact that the GPUs actually have locality with respect to the CPU sockets. This topology also is causing downstream affects with the fluxion scheduler and its testsuite. Depending on how the hwloc is read (either via flux-core's flux-hwloc or directly via the hwloc API), the resource topology changes (i.e., GPUs are children of the node versus the sockets). Also worth noting that the GPUs are children of the sockets when using the hwloc V2 API, so ignoring the group creates a significant difference in the topologies between hwloc versions. Solution: remove the call to ignore `HWLOC_OBJ_GROUP` so that on the Sierra/Lassen systems, the GPUs are children of the sockets. This also normalizes the resource topology across reading methods and hwloc versions. Now requesting a GPU on Sierra/Lassen can always be done with a `node->socket->gpu` jobspec. Related PR: flux-framework/flux-sched/pull/677
|
Command |
SteVwonder
force-pushed
the
hwloc-unignore-group
branch
from
9dd99b3 to
9464337
Compare
Codecov Report
@@ Coverage Diff @@
## master #3046 +/- ##
==========================================
+ Coverage 80.84% 80.87% +0.02%
==========================================
Files 270 270
Lines 42798 42796 -2
==========================================
+ Hits 34602 34613 +11
+ Misses 8196 8183 -13
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem: When the
HWLOC_OBJ_GROUPis ignored, the GPUs on theSierra/Lassen clusters are represented in the resource topology as
direct children of the node. This topology ignores the fact that the
GPUs actually have locality with respect to the CPU sockets. This
topology also is causing downstream affects with the fluxion scheduler
and its testsuite. Depending on how the hwloc is read (either via
flux-core's flux-hwloc or directly via the hwloc API), the resource
topology changes (i.e., GPUs are children of the node versus the
sockets). Also worth noting that the GPUs are children of the sockets
when using the hwloc V2 API, so ignoring the group creates a significant
difference in the topologies between hwloc versions.
Solution: remove the call to ignore
HWLOC_OBJ_GROUPso that on theSierra/Lassen systems, the GPUs are children of the sockets. This also
normalizes the resource topology across reading methods and hwloc
versions. Now requesting a GPU on Sierra/Lassen can always be done with
a
node->socket->gpujobspec.Related PR: flux-framework/flux-sched/pull/677