[META] Identity in OpenSearch

[DarshitChanpura]

Supporting Native AuthN/AuthZ with Core

Every thing done within OpenSearch should be permissible, no matter the source. This is why we need to move authentication and authorization into OpenSearch core codebase. The plugin model is very useful for optional things, security primitives are not one of them.

Getting these new native concepts out to OpenSearch developers and customers quickly for feedback and iteration is our aim to achieve this.

Terminology:

• Subject: An individual, process, or device that causes information to flow among objects or change to the system state.
• Principal: An application-wide identifier that is persisted

Minimal viable system

To move quickly, and provide a surface area for experimentation we are going to cut some corners. This effort will first be within a feature branch, then be controlled via an experimental feature flag shipped with OpenSearch distribution, finally made available for adoption with backwards compatible support.

• Ship with an experimental feature flag
• Introduce Identity as a module. PR [Feature/Identity] Introduce Identity module #5583

Blog Posts

This section is for tracking any updates that would be made Identity related blog-posts.

• Identity Blog Post PR Introducing Identity for OpenSearch blog post project-website#1149
• Introducing Identity use-cases PR [Feature/Identity] Identity use cases #5513

The following are the areas of focus to create this minimal set of functionality along with an estimated complexity to develop to help in terms of understanding the projects. Complexity attempts to score uncertainty, not effort, delivering more complex items implies more prototypes/design reviews.

Identity Concepts

• [Complexity 8] Introduce Identity primitives (Subject and Principal) and their associated objects and . See following issue for more details

• OpenSearch requests should know the identity of the caller #3846

• Add support for using User Identity in extensions #37

• Create a permissions check API for extensions to us with custom resources #40

Principal Identifiers

• [Complexity 2] Principal identity for Opensearch requests to/from extensions PR [Feature/Extensions] Principal identity for Opensearch requests to/from extensions #4299
• [Complexity 3] [Feature/Extensions] Follow-up: Implement Encryption/Decryption for principal identifier token #4485

Delegate Execution

• [Complexity 5] Strategize authority Delegation via JWT Tokens PR [Feature/Identity] Strategy for Delegated Authority using Tokens #4826
• [Complexity 5] Delegate any tasks to system subject or interactive user sessions. The default mechanism should use access tokens that are cryptographically secured against tampering.

Access Tokens

• [Complexity 5] Add APIs to: generate token for principal, refresh token, expire all refresh tokens
• [Complexity 5] Secrets generation / storage

Internal Identity Provider

• [Complexity 2] Create basic in memory user store for username/password based authentication PR [Feature/Identity] Prototype Internal IdP #4659
• [Complexity 3] Persistently store the users across the cluster in a way that is kept consistent PR [Feature/Identity] Persistence for the Identity module using a System Index #5613

CRUD operations on Subject

• [Complexity 3] Allow CRUD operations for subjects for system processes or for users to run on their behalf PR [Feature/Identity] User operations: create update delete #4741
• [Complexity 3] Add REST API for subject creation

Authentication

• [Complexity 3] Support basic username/password authentication PR [Feature/Identity] Adds Basic Auth mechanism via Internal IdP #4798
• [Complexity 3] Support bearer token authentication PR [FEATURE/IDENTITY] Add bearer authentication to feature/identity #5611

Permissions/Authorization

• [Complexity 3] Design the permissions model and any potential changes. Notability existing permissions based on action names will be on deprecation path PR [FEATURE/Identity] Outline Permissions Model and updated Permission Nomenclature #5698
• [Complexity 3] Adding Permissions check API [Feature/Identity] Permissions check API #4516
• [Complexity 2] Registration of permissions from all permissions sources in core
• [Complexity 5] Registration of permissions for extension/plugins
• [Complexity 2] Have a process where from a Subject a list of permissions is generated so they can be checked
• [Complexity 8] For the new permission come up with a structured naming convention building blocks, such as {SourceName}{ResourceName}{ActionVerb} -> Ad.Detector.Create. Additionally come up with a 'blessed' list of action verbs to prevent http methods from being used as action names.
• [Complexity 2] With the new naming convention create updated names for all core permissions
• [Complexity 2] Confirm evaluation of permissions for Actions in core
• [Complexity 2] Grant permissions to a subject, through the identity provider

TODOs

This section contains all the current and pending TODO items in the identity code.

• [Feature/Identity] TODOs introduced while developing identity project #5835

Future Scope

The following items are not required for the critical path delivery and will likely be started on following the stabilization of the MVP components.

Loading from configuration

• [Complexity 8] Reuse/replace existing YAML configuration

Index View(s)

• [Complexity 8] Might be further expanded into a generic 'resource collection'

Authorization Abstractions

• [Complexity 8] Roles, groups, trees, or other intermediate representations of permissions.

Rename plugin permissions (OOB?)

• [Complexity 8] Drive the renaming of permissions through all the existing plugins.

Migration from Security Plugin

• [Complexity 13] How to do a complete, or partial migration(s) from the security plugin.

Authentication providers

• [Complexity 8] OIDC, SAML, and other(s).

Security testing

• [Complexity 8] How to author test cases for security features, and building out these test cases for all critical scenarios

Identity Information

• [Complexity 8] Allow retrieve/storage of user information

[dblock]

This is quite thorough!

I'd be careful about saying "thing", and get more specific. So every thing done within resource and external api call in OpenSearch should be unambiguously permissible, nomatter the source and we also need to say something like the starting process needs to launch with a built-in identity. Every thread spawned, and every task scheduled should be running under a clear identity.

I think This is why we need to move authentication and authorization into OpenSearch core codebase. The plugin model is very useful for optional things, security primitives are not one of them. isn't that well explained. Users expect to be able to reason about access in an unambiguous way and the easiest way to do it is with the principle of least privilege, where by default nobody has access to anything. The easiest way to implement this is to move security behind APIs, ie. into core.