add LicenseState() to SystemView interface

[thyton]

Description

Currently, enterprise plugins can be only run as builtin plugins. To allow enterprise plugins to run as external plugins, we need a way to validate Vault license. This PR adds LicenseState() to SystemView so that external plugins can request license state over gRPC, following a similar work done in #24929.

The new method can replace the current approach of using addLicenseCallback() to assign the plugin's Backend.LicenseState(). Whether the plugin is run as a builtin plugin or an external plugin, the plugin backend can use System().LicenseState(). If the plugin is no longer an enterprise plugin, we can simply remove the license check on the plugin repo.

func (b *Backend) HandleExistenceCheck(ctx context.Context, req *logical.Request) (checkFound bool, exists bool, err error) {
	// check license state using vault-licensing then feature
	licenseState, err := b.Backend.System().LicenseState()
	if err != nil {
 	    return false, false, errutil.UserError{Err: "license check failed, rejecting request"}
        }

	isValidState := license.IsValidState(licenseState)
        if isValidState || b.Backend.HasFeature(license.Features(entlicense.FeatureKeyManagementSecretsEngine)) {
	    return false, false, errutil.UserError{Err: "license check failed, rejecting request"}
	}

	// handle existence check
}

func (b *Backend) HandleRequest(ctx context.Context, req *logical.Request) (*logical.Response, error) {
	// check license state using vault-licensing then feature
	licenseState, err := b.Backend.System().LicenseState()
	if err != nil {
 	    return false, false, errutil.UserError{Err: "license check failed, rejecting request"}
        }

	isValidState := license.IsValidState(licenseState)
        if isValidState || b.Backend.HasFeature(license.Features(entlicense.FeatureKeyManagementSecretsEngine)) {
	    return false, false, errutil.UserError{Err: "license check failed, rejecting request"}
        }

        // handle request
}

Test output shows Vault with new plugin sdk works with older plugins with previous plugin sdk

root@6fa90bb39cbf:/vault_1.18.0# uname -a 
Linux 6fa90bb39cbf 6.6.31-linuxkit #1 SMP Thu May 23 08:36:57 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
root@6fa90bb39cbf:/vault_1.18.0# ./vault version
Vault v1.18.0-beta1 ('4b69030a147e84baa31b2b8d0db4784d5f430d08'), built 2024-08-12T10:20:51Z

root@6fa90bb39cbf:/vault_1.18.0# sha256sum ../plugins/vault-plugin-secrets-kv 
cd64c98f2695e13c4364650b53cb51406e699ee5fc2db67025f669f5d4c5c5d6  ../plugins/vault-plugin-secrets-kv
root@6fa90bb39cbf:/vault_1.18.0# ./vault plugin register -sha256=cd64c98f2695e13c4364650b53cb51406e699ee5fc2db67025f669f5d4c5c5d6 -version=v0.17.0 secret vault-plugin-secrets-kv
Success! Registered plugin: vault-plugin-secrets-kv

root@6fa90bb39cbf:/vault_1.18.0# ./vault plugin list | grep secrets-kv
vault-plugin-secrets-kv              secret      v0.17.0
root@6fa90bb39cbf:/vault_1.18.0# ./vault secrets enable -path=my-secret vault-plugin-secrets-kv    
Success! Enabled the vault-plugin-secrets-kv secrets engine at: my-secret/

root@6fa90bb39cbf:/vault_1.18.0# ./vault secrets list -format=json
...
  "my-secret/": {
    "uuid": "80e47a38-1c89-1d86-abc6-d088ceb759af",
    "type": "vault-plugin-secrets-kv",
    "description": "",
    "accessor": "vault-plugin-secrets-kv_b99ecf5f",
    "config": {
      "default_lease_ttl": 0,
      "max_lease_ttl": 0,
      "force_no_cache": false
    },
    "options": null,
    "local": false,
    "seal_wrap": false,
    "external_entropy_access": false,
    "plugin_version": "v0.17.0",
    "running_plugin_version": "v0.17.0",
    "running_sha256": "cd64c98f2695e13c4364650b53cb51406e699ee5fc2db67025f669f5d4c5c5d6",
    "deprecation_status": ""
  },
...

TODO only if you're a HashiCorp employee

• Labels: If this PR is the CE portion of an ENT change, and that ENT change is
  getting backported to N-2, use the new style backport/ent/x.x.x+ent labels
  instead of the old style backport/x.x.x labels.
• Labels: If this PR is a CE only change, it can only be backported to N, so use
  the normal backport/x.x.x label (there should be only 1).
• ENT Breakage: If this PR either 1) removes a public function OR 2) changes the signature
  of a public function, even if that change is in a CE file, double check that
  applying the patch for this PR to the ENT repo and running tests doesn't
  break any tests. Sometimes ENT only tests rely on public functions in CE
  files.
• Jira: If this change has an associated Jira, it's referenced either
  in the PR description, commit message, or branch name.
• RFC: If this change has an associated RFC, please link it in the description.
• ENT PR: If this change has an associated ENT PR, please link it in the
  description. Also, make sure the changelog is in this PR, not in your ENT PR.

[github-actions]

CI Results:
All Go tests succeeded! ✅

[github-actions]

Build Results:
All builds succeeded! ✅

[fairclothjm]

Is it possible for the caller to pass the context instead of using context.Background() here?

[fairclothjm]

On the one hand, I like this approach because it is simple. It does not require us to update https://github.com/hashicorp/vault-licensing and other places in Vault core's code to add a new ent plugin.

On the other hand, it sidesteps the current paradigm of explicitly allowlisting each feature/plugin. So I am not sure about this. I am having trouble coming up with any downsides to your current approach other than we don't have fine-grained control. Maybe we should get more feedback from our team?

[thyton]

it sidesteps the current paradigm of explicitly allowlisting each feature/plugin

I agree. Along this line, I wonder, if any point we have different types/tiers of licensing, HasLicense() will need to be changed. This makes me think HasFeature() of SystemView interface, currently left unimplemented, could solve the fine-grained control in both situations.