Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[develop] upgrade slurm #5695

Merged
merged 1 commit into from
Oct 6, 2023
Merged

[develop] upgrade slurm #5695

merged 1 commit into from
Oct 6, 2023

Conversation

NSsirena
Copy link
Contributor

@NSsirena NSsirena commented Sep 12, 2023

Description of changes

Porting in develop of the fix about PMIx and the slurm upgrade made in Release-3.7 branch.

After discussing the PR internally we decided to remove the version check from the integration tests since we already test and validate the versions in the Cookbook package.
This will also avoid and extra PR for each version upgrade.

Tests

  • See linked PR

References

Checklist

  • Make sure you are pointing to the right branch.
  • If you're creating a patch for a branch other than develop add the branch name as prefix in the PR title (e.g. [release-3.6]).
  • Check all commits' messages are clear, describing what and why vs how.
  • Make sure to have added unit tests or integration tests to cover the new/modified code.
  • Check if documentation is impacted by this change.

Please review the guidelines for contributing and Pull Request Instructions.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@@ -1749,7 +1749,7 @@ def _gpu_resource_check(slurm_commands, partition, instance_type, instance_type_
def _test_slurm_version(remote_command_executor):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we remove this slurm version check from the CLI and keep it just in the kitchen tests?
This will avoid us to create a patch for the CLI everytime we're updating slum version.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I completely agree!

We discussed this many times and for a reason or another always postponed the removal of this check.

I'll take this opportunity to do it.

Copy link
Contributor Author

@NSsirena NSsirena Sep 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And I'll go even further and remove also this check

# Since we're installing PMIx v4.2.6, we expect to see pmix and pmix_v4 in the output.
    ...
    assert_that(mpi_list_output).matches(r"\s+pmix_v4($|\s+)")

since we are checking and validating this in the cookbook.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may still need the patch for the Changelog.

@codecov
Copy link

codecov bot commented Sep 12, 2023

Codecov Report

All modified lines are covered by tests ✅

Comparison is base (f1974ba) 89.95% compared to head (9ab0c18) 89.95%.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop    #5695   +/-   ##
========================================
  Coverage    89.95%   89.95%           
========================================
  Files          180      180           
  Lines        15526    15526           
========================================
  Hits         13966    13966           
  Misses        1560     1560           
Flag Coverage Δ
unittests 89.95% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@NSsirena NSsirena added the skip-changelog-update Disables the check that enforces changelog updates in PRs label Oct 6, 2023
@NSsirena NSsirena enabled auto-merge (rebase) October 6, 2023 13:47
Comment on lines 268 to 281
# Ensure the expected PMIx version is listed when running `srun --mpi=list`.
# Since we're installing PMIx v3.1.5, we expect to see pmix and pmix_v3 in the output.
# Sample output:
# [ec2-user@ip-172-31-33-187 ~]$ srun 2>&1 --mpi=list
# srun: MPI types are...
# srun: none
# srun: openmpi
# srun: pmi2
# srun: pmix
# srun: pmix_v3
# srun: pmix_vX
#
# _vX is the Major number of the PMIx version installed and used to compile slurm.
# We check this in the cookbook, so we do not repeat the check here
mpi_list_output = remote_command_executor.run_remote_command("srun 2>&1 --mpi=list").stdout
assert_that(mpi_list_output).matches(r"\s+pmix($|\s+)")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say that these are also canary-like tests and we don't have to test them here. But that's probably something for the future.

@NSsirena NSsirena merged commit 8eb44dd into aws:develop Oct 6, 2023
26 of 27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.x skip-changelog-update Disables the check that enforces changelog updates in PRs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants