Investigate and define the work required for SPIFFE-based client identity management

[paulhowardarm]

Summary

This request is to establish a design and create the necessary work items (in this repo and other repos) that are needed to enhance Parsec with a SPIFFE-based identity subsystem.

Details

Investigation required, but necessary work items likely to include:

• Creating a new authenticator module in the service that can verify SPIFFE IDs. The expectation is that we will use the JWT SVID in the auth header of the request, with a new integer auth type, and that verification would be performed by calling the workload API ValidateJWTSVID operation to check the JWT and return the SPIFFE identity string (which is then used as the keystore namespace in the familiar way).
• Creating a branch in the client library/libraries to populate the auth field with a JWT SVID when the service requires this auth type.
• Creating an appropriate integration test system.
• Documentation updates.

Dependencies

At the moment, the SPIFFE spec does not allow clients to call ValidateJWTSVID without themselves being a registered SPIFFE workload. The expectation is that Parsec would not be a workload. However, the SPIFFE specification group have agreed that this restriction can be lifted. See spiffe/spiffe#132

Once the spec is changed, SPIRE (as the reference implementation) will also need to honour that change so that Parsec can call the function on a SPIRE-based system. There isn't a GH issue for that yet.

There will be a need to call SPIFFE workload API operations from Rust. We need to investigate whether this can be done by consuming https://github.com/heavypackets/rust-spiffe and adding the necessary JWT operations (since it currently only supports X509).

Definition of Done

This issue can be considered done once we have created a credible set of well-understood issues in this repo and in the other repos for an end-to-end SPIFFE multi-tenant solution. We should by this point be in a position for any contributor(s) to start implementing those pieces without requiring further investigation.

[ionut-arm]

We might need to collaborate with Sabree on this one to use his spiffe crate or to get approval to fork it and upstream.

[paulhowardarm]

As part of this investigation, we also need to work with the SPIFFE and SPIRE project representatives to decide on what a proof-of-concept or demo milestone should look like. When expanding this issue into separate engineering items, we should include the work needed to build a demo or showcase of some kind. One example might be to extend the command-line client so that we can demonstrate the creation of keys and show that different workloads cannot "see" each others' keys.

[hug-dev]

Work needed to complete this task entirely:

1. add in the rust-spiffe crate support for JWT-SVID (estimate: 2 weeks). More precisely:
   a. the ability to fetch its JWT-SVID
   b. the ability to validate a JWT-SVID remotely (faster, for PoC/demos)
   c. the ability to validate a JWT-SVID locally (for security, will take longer)
2. add a new AuthType in Parsec Interface (Done): Add a JWT SVID authentication type parsec-interface-rs#84
3. add a new authenticator in Parsec to validate a JWT-SVID (estimate: 1 week): Add a JWT-SVID Authenticator #268
4. add a new variant to the Rust client's AuthenticationData for JwtSvid and a constructor that uses rust-spiffe to automatically fetch the JWT-SVID it will use to authenticate (estimate: 1 week): Add a JWT-SVID authentication data parsec-client-rust#55
5. add a test to the multitenancy testing infrastructure for SPIFFE authentication on the CI (estimate: 1 week): Add a SPIFFE JWT-SVID multitenancy test #269
6. update the relevent pages in the book with all of those changes (estimate: 1 week): Update with the new JwtSvid authenticator parsec-book#72

In terms of doing a demo, the following could be done locally and recorded (can be done before any PR is up):

• two workloads as different Unix users get registered to the SPIRE server based on their UID
• on two different terminals, the two workloads create a key on the Mbed Crypto provider, selecting the JwtSvid authentication method. That assumes that 1.a, 2 and 4 are done.
• Parsec validates the JwtSvid and creates the key. That assumes that 1.b and 3 are done.
• the clients list their keys: they can only see the one they created
• the clients try to destroy the other's client key: it does not work
• the clients list their keys: same output
• the clients destroy their own key
• the clients list their keys: empty result

[hug-dev]

@paulhowardarm I will let you close this issue if you think the results of the investigation are satisfactory.