libjob: need signing interface when not configured --with-flux-security

[garlick]

The current API for job submission requires that users know whether flux-core was built --with-flux-security or not. This will not work for external API users. In addition, it complicates the python bindings as discussed in #1709.

If security is unavailable, currently jobspec is submitted (and stored) in the clear without post processing, e.g.

/* Submit a job to the system.
 * J should be RFC 14 jobspec signed by flux_sign_wrap(), provided
 * flux was built --with-flux-security.  If not, then J should be bare jobspec.
 * Currently the 'flags' parameter must be set to 0.
 * The system assigns a jobid and returns it in the response.
 */
flux_future_t *flux_job_submit (flux_t *h, const char *J, int flags);

It would be trivial to create a signed jobspec using mech=none that could be parsed by flux security, without duplicating code. The format is HEADER.PAYLOAD.SIGNATURE where

• HEADER is a base64 encoding of the header (see below)
• PAYLOAD is a base64 encoding of the payload
• SIGNATURE is the string "none"

The header is a kv string (see src/util/kv.h in flux-security)

userid\0i<uid>\0mech\0snone\0version\0i1\0

where <uid> is the signing userid.

The reverse (unwrap) would also be necessary in the ingest module.

I guess the more difficult question is what sort of API should be exported by flux-core for signing that works with and without --with-flux-security. When built "with", command line tools need a way to override the default flux-security config file for testing, and to override the configured signing mechanism. Currently one must call something like:

flux_security_t *sec;
const char *signed_string;

sec = flux_security_create (0);
flux_security_configure (sec, "path/to/config");
signed_string = flux_sign_wrap (sec, inbuf, inbuf_size, "signtype", 0);
...
flux_security_destroy (sec);

Maybe (as @SteVwonder was suggesting I think) a generic wrap/unwrap could be added to the flux_sec_t context already in flux-core? That's sounding like a better idea now because it would provide a way to hold the flux-security context (optionally).

[SteVwonder]

Maybe (as @SteVwonder was suggesting I think) a generic wrap/unwrap could be added to the flux_sec_t context already in flux-core?

I think credit goes to you for that idea. At the surface, it seems like a good idea. Unfortunately, I don't know enough about the two interfaces to really say. Sorry.

[garlick]

Another thought - are the config/mech options even useful outside of flux-core (outside of testing really)? could we add a FLUX_JOB_SIGNED flag to flux_job_submit() to allow testing with various security mechanisms in tree, and if not flag not present, sign using a security context initialized on demand and cached in the flux_t handle?

[grondo]

I guess the more difficult question is what sort of API should be exported by flux-core for signing that works with and without --with-flux-security. When built "with", command line tools need a way to override the default flux-security config file for testing, and to override the configured signing mechanism.

There is something that seems slightly off about the current interfaces. Ideally, flux-core could have a generic sign and verify routines that do not expose bits of the underlying mechanisms to the caller, for the 99% of cases where the caller doesn't want to (or shouldn't) choose a mechanism (nor an explicit config file for that matter)

I think there may be some tricky corner cases, but the generic core APIs should be able to auto-configure themselves by reading a system config file, or perhaps via loading a set of implementations from well defined system paths. Optionally the default configuration could be overridden via a flux_security_configure() or flux_security_set_sign_type() to set the default signature mechanism for future calls to sign.

Maybe (as @SteVwonder was suggesting I think) a generic wrap/unwrap could be added to the flux_sec_t context already in flux-core?

Just my opinion, but that doesn't sound right to me. In fact, that flux_sec_t API doesn't even seem like it should be publicly exported.?

[grondo]

Another thought - are the config/mech options even useful outside of flux-core (outside of testing really)?

Oh, yeah. That is kind of what I was trying to get at (though less eloquently)