[Extensions] Add Transport API for fetching Service Account information from security plugin

[stephen-crawford]

In order to implement Service Accounts (#2597), it needs to be possible to fetch service account details. To do this, OpenSearch core should be able to call the Security Plugin to provide a Service Account Authorization Header back to the Extensions Manager in core. This will let core then verify the operations of the associated extension.

The process should follow

sequenceDiagram
    participant A as Extension_1
    participant B as Core ServiceAccountManager
    participant C as Security Plugin
    participant D as InternalUsers
    participant E as Response
    participant F as Create ServiceAccount

    A->>B: Install
    B->>C: getOrCreate ServiceAccount
    C->>D: Check ServiceAccount
    alt ServiceAccount Exists
        D->>E: Return ServiceAccount Info
    else ServiceAccount does not exist
        D->>F: Create ServiceAccount
        F->>D: Store ServiceAccount
        D->E: Return ServiceAccount Info
    end
    E-->>B: ServiceAccount Info

Loading

The alt and else blocks show the decision point where the process either checks if the service account exists or creates it if it does not exist.

The general implementation process will require adding a transport layer API to the Security Plugin so that Core can call the API with a provided extensionId string. This can take the form of an API stack in a package located at src/main/java/org/opensearch/security/extensions/api/. Since we are only concerned with getting the Service Account credentials, the plan is to combine both the createServiceAccount and getServiceAccount actions into one. Basically, we want a single action ServiceAccountGetAction to provide service account auth credentials regardless if a service account was created for that specific extension before or not.

When the request reaches the Security Plugin, the Security Plugin should look in the InternalUsers for a matching Service Account (denoted by a name which equals the extensionUniqueId and with an attribute service:true). If one exists, the Security Plugin should generate an auth token to return to OpenSearch core based on the information in the InternalUsers store (this is a separate issue, for now we can just return a string saying "account found" or something).

If no matching Service Account is found, then the Security Plugin will need to create a new Service Account for the extension and store it in the InternalUsers Storage. Again, this will be a separate issue, so for now it is sufficient to return "no account found, new service account created" etc. This response should be passed back to OpenSearch core so that the Extensions Manager will ultiamtely be able to parse the message and act on its content.

Completion of this issue will look like a PR which adds the required classes to the ../extensions/api/ package that would allow OpenSearch core to provide an extensionId and either get the matching service account credentials or create a new service account based on that ID and then return the new credentials. The PR will need to test that the API accepts and responds appropriately to a mocked request from core but does not need to make any changes in core. Likewise, Service Account lookup, auth token creation, and service account creation are outside the scope of this issue. You can make this PR against the main branch if that seems appropriate or alternatively against the feature/identity branch if the PR does not seem self-contained enough that merging to main would be appropriate (generally, we only want tested, and isolated changes in main). You can reach out to @scrawfor99 on this issue or on the OpenSearch slack for questions about the PR completion.

A basic starting point for this comes from the refactor of the user system. Specifically #2594, which introduces a user service for handling service accounts and internal users.

Eventually, inside Core, a hook will need to be added that allows the API to be called when a new extension is registered with the Extension Manager. An outline of what this may look like can be found here. This may be helpful when completing this issue.

[MaciejMierzwa]

hi, @scrawfor99. Just for clarification regarding this task, are we looking to add an API registered as RestHandler or a node-to-node API for internal communication?

[stephen-crawford]

Hi @MaciejMierzwa, so sorry for the confusion. I ended up modifying the existing model of the Security Plugin as part of #2594. That made this task unnecessary since we can now use the existing query paths. I will close this issue out so that there is no confusion. Apologies again.