WIP: job-manager: add job validation #3192
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add job-manager.validate RPC that can look up a job ID and return the basic info that the job manager keeps for it: userid - job owner t_submit - submit timestamp priority - job priority state - job state flags - submit flags Inactive jobs are handled too. If the job is not found in the job manager's active job hash, its eventlog is fetched from the KVS and replayed. If the KVS lookup fails with ENOENT, the RPC returns with ENOENT and an "invalid job ID" human-readable message. This response is definitive since the submit RPC does not return a job ID to the user until the job manager is aware of it. This RPC is currently open to guests. Fixes flux-framework#3191
Add 'flux job validate ID'. This command just indicates whether the ID is valid through its exit code. If there is a failure, a message is displayed on stderr.
Codecov Report
@@ Coverage Diff @@
## master #3192 +/- ##
==========================================
- Coverage 81.24% 81.12% -0.13%
==========================================
Files 295 296 +1
Lines 44456 44541 +85
==========================================
+ Hits 36120 36133 +13
- Misses 8336 8408 +72
|
|
at a basic level looks good. I tried to see if it could be seamlessly dropped into it should speed up the code on average, by reducing the number of eventlog lookups on active jobs. There would be cases that are slower b/c we'd be doing lookups of the eventlog twice (i.e. user lookups eventlog of inactive job, jobid ccheck does an eventlog lookup too). |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Here's a first cut at a
job-manager.validateRPC as discussed in #3133This implementation handles inactive jobs by replaying a job's KVS eventlog if it's already been purged from the job manager's active job hash. Thus the response can definitively identify an invalid job ID.
The basic job info that job-manager keeps is returned (t_submit, priority, userid, flags, state). The request is open to guests (hopefully that is OK?)
I also added a
flux job validate IDcommand. It doesn't display any of the returned info but probably it should at least have an option to do that.Thought I'd park it as a WIP to get some feedback before proceeding with tests.