Enforce size constraints #202
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Problem: as rabbits enter more general use, there needs to be a way to enforce crude limits on the size of file systems that users can request. Add command-line arguments to coral2_dws for accepting size limits and logic to the directivebreakdown module for enforcing the limits.
Problem: the test suite creates a nnfstorageprofile and then deletes it, but if the test fails the profile is not deleted and subsequent runs will fail because the profile is not supposed to exist when the tests start. Make sure to delete the profile before starting the tests.
Problem: the coral2_dws script sends a resource-update RPC with a non-null 'errmsg' field if a job requests storage that is beyond the size constraints for file systems specified on the coral2_dws command line. However, the dws-jobtap plugin does not act on those error messages. Add logic to the jobtap plugin to raise an exception if the errmsg field is not null.
Problem: the sham coral2_dws script used by the testsuite does not send errmsg fields with its resource-update RPCs. Add the errmsg field.
Problem: the sham coral2_dws script in the test directory is poorly formatted. Format it with black.
cmoussa1
approved these changes
Aug 27, 2024
| -N1 -n1 hostname) && | ||
| flux job wait-event -vt 10 -m description=${CREATE_DEP_NAME} \ | ||
| ${jobid} dependency-add && | ||
| flux job wait-event -vt 20 ${jobid} exception && |
There was a problem hiding this comment.
Super minor and I'll leave this up to you but it might be nice to check to see that the exception raised on these failed job submissions matches what you expect? i.e grep for the exception message raised
There was a problem hiding this comment.
Good call, done and force-pushed. Will set MWP.
Problem: there are no tests to ensure the enforcement of storage constraints works. Add tests.
Problem: attempting to remove a dependency after raising an exception on the job fails, and is unnecessary. Return after raising the exception.
jameshcorbett
force-pushed
the
enforce-size-constraints
branch
from
a70fd3b to
d11e871
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem: as rabbits enter more general use, there needs to be a way
to enforce crude limits on the size of file systems that users can
request.
Add command-line arguments to coral2_dws for accepting size limits
and logic to the directivebreakdown module for enforcing the limits.
The limits are enforced by adding an exception with a message while the job is held in DEPEND state.