Add gRPC definitions for participant user management service

[meiersi-da]

This PR introduces the gRPC definitions for an experimental user management service to manage users and their rights for interacting with the Ledger API served by a participant node. The PR does not change the set of actually exposed services of a Ledger API server.

I expect to implement the user management service in a later PR behind a feature flag such that we can start a PoC phase to evaluate whether all the Daml SDK tools can be made to work well (or hopefully better) building on this user management service. I expect that we'll iterate further on the service's gRPC definitions as part of this PoC phase.

@bame-da @cocreature @stefanobaghino-da could I ask one of you to approve the PR so we can start the PoC implementation work?

FYI: @da-tanabe @adriaanm-da @gerolf-da and others, feel free to review the code directly on this PR and add comments here. I will then take care of addressing the comments on a new PR in case this one is already merged.

Pull Request Checklist

• Read and understand the contribution guidelines
• Include appropriate tests
• Set a descriptive title and thorough description
• Add a reference to the issue this PR will solve, if appropriate
• Include changelog additions in one or more commit message bodies between the CHANGELOG_BEGIN and CHANGELOG_END tags
• Normal production system change, include purpose of change in description
• If you mean to change the status of a component, please make sure you keep the Component Status page up to date.

NOTE: CI is not automatically run on non-members pull-requests for security
reasons. The reviewer will have to comment with /AzurePipelines run to
trigger the build.

[da-tanabe]

I largely agree with these numbers, but given #10283 this could be a tough API to change later…

[meiersi-da]

My aim would be to add pagination support as per https://cloud.google.com/apis/design/design_patterns#list_pagination

This can be done in a backwards compatible way, by assuming a default limit in case none is specified.

[bame-da]

All makes sense to me. My main question is why you are designing this as extra services rather than just keeping users and rights on-ledger.

template User
  with
    participant : Party
    userParty : Party
    userName : Text
  where
    ensure (valid userName)
    signatory participant
    observer userParty

data Right
  = Admin
  | ReadAs Party
  | ActAs Party

template UserRight
  with
    participant : Party
    user : Party
    right : Right
  where
    signatory participant
    observer user

Now the API is given by the transaction, acs, and command APIs, and the authorization is dealt with in the Daml Model.

[bame-da]

If we do this in Daml there's a question of causality and transactionality. Ie I could imagine two models:

1. The user rights are resolved once at the beginning of command processing. So the following sequence of events is valid
   A. Admin submits TX1: Delete User Alice
   B. Alice Submits TX2
   C. TX1 completes
   D. TX2 completes
2. The user rights are resolved as part of validation and can change during a transaction. So the above is invalid, but the following is valid, allowing for "transient" user rights:
   A. Alice submits a transaction with events
   1. Exercise delegated choice to add ReadAs right for user Alice on Party Bob
   2. Fetch contract known only to Bob
   3. Exercise delegated choice to remove ReadAs right for user Alice on Party Bob

When I talk about modelling users in Daml, I have model 1 in mind for now.

[meiersi-da]

All makes sense to me. My main question is why you are designing this as extra services rather than just keeping users and rights on-ledger.

@bame-da: I agree that we ultimately want to end up with using Daml for managing the participant-local state as well. The main motivation for introducing this API is to manage technical risk. The API proposed here is simple and boring, and thus has a low chance of unforeseen implementation problems.

When building user management directly in Daml, we for example have to consider how to upgrade the User contracts in the future to add new features without breaking all clients interacting with the current version. Likewise, there is a potential bootstrapping problem for creating the first user contract that allocates the 'admin' user with admin rights. I believe we can and should discover and solve these problems, but not on a tight deadline.

Instead, I propose that we introduce a simple gRPC API that

1. covers the needs on which we have a tight deadline, and
2. can be implemented both with direct DB access and also using on-ledger Daml contracts.
   This way we can first deliver the API using direct DB access, and then iterate on it in a backwards-compatible way using on-ledger Daml contracts as the backing storage.

What do you think?

[bame-da]

What do you think?

Makes sense, you are almost certainly right that this is the lower-risk route.

[cocreature]

Thank you!

The API looks very reasonable. So far, I think I’m fairly confident we can implement this for 2.0.

My main concern is the amount of breakage we might cause for our users and our examples if we try to change e.g., Daml Script to work with users by default. But that definitely doesn’t need to block this PR and I also haven’t thought too deeply about it so far.

[stefanobaghino-da]

I'll make sure to find time to review the design and this PR by the end of the day.

[gerolf-da]

cc @mziolekda

[gerolf-da]

Is there a chance that we want to transmit other metadata than what is contained in User?

[meiersi-da]

@gerolf-da : I'm following the Google API Style guidelines here: https://cloud.google.com/apis/design/standard_methods

If we really require more info to be transmitted, then we should introduce a custom method: https://cloud.google.com/apis/design/custom_methods

[gerolf-da]

Maybe it's useful to also include all rights (or just the set of already existing rights) in the response. If I think about the usage of this in a GUI, I'd probably want to refresh the user's rights with a definitive authoritative answer about all rights the user has.

[meiersi-da]

The reason why these rights are not included is that this won't scale to large sets of rights.

[stefanobaghino-da]

Sorry, couldn't review today, I'll need to do it tomorrow.

[cocreature]

In Daml script, the natural thing to do seems to be to flip this around immediately and turn it into:

data Rights = Rights
  { actAs : [Party]
  , readAs : [Party]
   , admin : Bool
   }

I could see other clients wanting to do the same thing (the JSON API for example). Have you considered emitting the result in something closer to that format?

[meiersi-da]

I have tried it, and it was not easy to present that way. The flat set of rights is a simple, well-defined structure that lends itself well to extensions.