licensing: clarify the multi-tenant UX of setting the license cluster setting

[knz]

Describe the problem

Currently, our enterprise license configuration UX is as follows:

• an operator would set the license using SET CLUSTER SETTING enterprise.license on the system tenant.
• a metric called seconds_until_enterprise_license_expiry is exported at the prometheus endpoint for the system tenant
• in every secondary tenant server, the enterprise license setting is left empty unless the environment variable (not cluster setting) COCKROACH_TENANT_LICENSE is set to contain a valid license.
  • this env var is set in CC serverless.

This is problematic for our multi-tenancy plans, where we want Dedicated/SH customers to use secondary tenants by default.

As of this writing, users trying to point their apps to CockroachDB will encounter errors relating to a missing Enterprise license.

The reason why the Eng team did not see this is because our dev workflow auto-enables licensing in other ways

• cockroach demo disables license checks globally
• most team members have the env var COCKROACH_DEV_LICENSE set in their shell to override the license

Expected behavior

TBD. A strawperson argument:

• eventually, it will be possible for separate tenants to be differently licensed. We may want to make the license setting per-tenant, with a default inherited from the env var COCKROACH_TENANT_LICENSE
• in shared-process multitenancy, we may want a way to auto-propagate the license from the system tenant to all secondary tenants running in the same process.

Further discussion is needed.

Jira issue: CRDB-24012
Epic: CRDB-26689