Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MALI update to fix issues from earlier PR causing sporadic test failures #6627

Merged
merged 1 commit into from
Sep 24, 2024

Conversation

jonbob
Copy link
Contributor

@jonbob jonbob commented Sep 19, 2024

Including a variable that was deactivated in the globalStats stream caused sporadic failures during the second run of some ERS tests on several platform/compiler combinations. That variable is now only included when MALI is using Albany. Also updates a namelist default that had been missed but does not change answers.

Fixes #6600

[NML] for configurations with MALI
[BFB]

@jonbob jonbob self-assigned this Sep 19, 2024
Copy link

PR Preview Action v1.4.8
🚀 Deployed preview to https://E3SM-Project.github.io/E3SM/pr-preview/pr-6627/
on branch gh-pages at 2024-09-19 16:28 UTC

@jonbob
Copy link
Contributor Author

jonbob commented Sep 19, 2024

@matthewhoffman -- I had also tested with adding a default value for albanyVelocityError in Registry. I don't think it makes any difference, but we could add it here if you want

@jonbob
Copy link
Contributor Author

jonbob commented Sep 19, 2024

It passed the failing ERS.f09_g16_g.MALISIA.pm-cpu_gnu three times in a row with these changes

@jonbob
Copy link
Contributor Author

jonbob commented Sep 19, 2024

Also ran ERS_Ld5.TL319_oQU240wLI_ais8to30.MPAS_LISIO_JRA1p5.pm-cpu_gnu.mpaso-ocn_glcshelf successfully three times with no fails. That test has also been having random problems since PR #6559

Copy link
Contributor

@matthewhoffman matthewhoffman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jonbob , looks good to me. Thanks for tracking this down. You are welcome to add the addition of the default_value for albanyVelocityError to Registry. Making the default default value explicit in this case isn't a bad idea.

@jonbob
Copy link
Contributor Author

jonbob commented Sep 19, 2024

Thanks @matthewhoffman -- OK, I've added the default value as well

@ndkeen
Copy link
Contributor

ndkeen commented Sep 19, 2024

Sounds like you've solved it. You might also try with more MPI's. Maybe 1024?

@rljacob rljacob added this to the v3.0.1 milestone Sep 23, 2024
jonbob added a commit that referenced this pull request Sep 23, 2024
MALI update to fix issues from earlier PR causing sporadic test failures

Including a variable that was deactivated in the globalStats stream
caused sporadic failures during the second run of some ERS tests on
several platform/compiler combinations. That variable is now only
included when MALI is using Albany. Also updates a namelist default
that had been missed but does not change answers.

Fixes #6600

[NML] for configurations with MALI
[BFB]
@jonbob
Copy link
Contributor Author

jonbob commented Sep 23, 2024

Passes, with expected NML DIFFs:

  • e3sm_landice_developer on chrysalis with the gnu compiler

Successfully runs:

  • ERS_Ld5.TL319_oQU240wLI_ais8to30.MPAS_LISIO_JRA1p5.pm-cpu_gnu.mpaso-ocn_glcshelf
  • ERS.f09_g16_g.MALISIA.pm-cpu_gnu

which had been failing

Merged to next

@jonbob jonbob merged commit 39d5295 into master Sep 24, 2024
5 checks passed
@jonbob jonbob deleted the jonbob/mali/fix-globalstats-list branch September 24, 2024 15:39
@jonbob
Copy link
Contributor Author

jonbob commented Sep 24, 2024

merged to master and expected NML DIFFs blessed -- plus regular DIFFs for the two tests fixed by this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Runtime errors with ERS.f09_g16_g.MALISIA
4 participants