Add role-based access control

[dongahn]

As part of PR #481, @SteVwonder mentioned:

Someone more knowledgeable than me, please correct me if I'm wrong, but after some digging, it looks like, by default, only messages sent by the instance owner will cause a message to be handled by the callback. You have to specifically add the FLUX_ROLE_USER rolemask when registering the callback to enable messages from other users.

Source: flux-framework/flux-core#1260 & garlick/flux-core@009b7ea

I initially thought the default was the opposite. So it looks like nothing needs to be done here in terms of security/handling, but the get_userid function will only ever return the instance owner's id.

This is to create role.based control within both qmanager and resource.

[garlick]

It's correct that the default behavior is to reject requests that do not have the OWNER role. All messages from the job manager to the scheduler will have the OWNER role.

This is to create role.based control within both qmanager and resource.

Happy to help with the design if you'd like to explain the use case. Is this to enable CLI tools used by guests to send administrative requests to the scheduler?

[dongahn]

Thanks @garlick.

This was created to log a loose end @SteVwonder and I identified as part of the PR and in fact to call out some use cases as well. There will be multiple use case:

• For the system instance scheduler, a non instance owner would want to query the resource states. For example find out about the reason for pending and estimated start time etc for their jobs as well as some other users' jobs.

• Your CLI use case is another...

I can see a general scheme where these front end commands are generally first coming to flux-core which then get authenticated and sent to the sched components. In a sense, free/exception request from jobmanager would be one such an example. IMO this cheme will simplify things a lot within sched.

In a sense our scheduler will "implement" the interface dictated by the flux-core for the requests that have been already authenticated and thus belong to the instance owner.

But then there will be likely other use cases specific to this scheduler, which are not covered by such an approach. Those are the ones tha would be supported through role based control within the sched itself if needed?

[garlick]

For the system instance scheduler, a non instance owner would want to query the resource states. For example find out about the reason for pending and estimated start time etc for their jobs as well as some other users' jobs.

See schedutil_alloc_respond_note() for one possible avenue for this. There is a placeholder in the protocol for the scheduler to update a brief, human-readable message for pending alloc requests (as many times as it wants). The thought was start time or other messages of interest to the submitter.

The job manager ignores these right now but it would be trivial to update the job manager and the queue lister to track and display them.

[dongahn]

Oh. This sounds pretty promising. Yeah the more the scheduler can push such job info to jobmanager and have it deal with user facing, the better the separation of concerns...

[grondo]

Random thought: We could also expand the human readable note to include an optional json dictionary, the keys and values of which could not only be used by query and other tools to filter and search on queued jobs, but this data could be returned to the scheduler during free and hello protocol to be used for internal purposes.

[dongahn]

@grondo: yes, I like this line of thinking a lot as it extend note into a more generalized solution. This way, job-manager can use the note field to display whatever the scheduler thinks is important for the users and the scheduler/other tools can use other fields for other purposes. Also, I like this approach more than the scheduling pushing info directly into kvs as we don't have to worry synchronization and etc.

[dongahn]

I will close this w/ the recent annotation API improvements. The only non owner RPC is sched.resource-status proposed as part of PR #694.