Reorganize Configuration section of the Security TOC

[cwillum]

Signed-off-by: cwillum [email protected]

Description

Break out authentication backend topics in the Configuration section into their own section to clean things up and reorganize in a more logical manner.

Issues Resolved

Created a new "Authentication backends" section to contain the specific backends that were previously strewn in with straightforward configuration topics.
Also addressing some issues in topics to improve clarity and gaps.
Addresses issue #2155.

Checklist

• By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
  For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[cwillum]

@opensearch-project/security Team, I reorganized the TOC for Security documentation. The biggest difference is that I broke out the authentication backends and separated them from general configuration information so they have their own section (more of that to come in a separate issue). @cwperks has seen the reorg.
My ask is if you'd have a quick look at some language I changed at the intro/top of four sections (in the diffs). Just to be sure it makes sense. I'll list those here. Then you don't have to wade through the many minor TOC changes. Thanks.
_security-plugin/authentication-backends/authc-index.md
_security-plugin/configuration/security-admin.md
_security-plugin/configuration/system-indices.md
_security-plugin/configuration/yaml.md

[cwillum]

@Naarcha-AWS Changes to the Security TOC are ready. I'll have you take a look first then pass it to another team member. Big thanks.

[cwperks]

Thank you @cwillum. I like breaking up the different authentication backends into menu items. I think that helps with discovery and navigation on the documentation website.

I left a couple of comments that I think can be addressed in future PRs as we work to make the documentation more precise and easy-to-follow.

[cwperks]

Is there any automation in place to warn against the use of specific version urls?

[cwillum]

I believe we're using an automated link checker now. Not sure if it would identify this specific problem. But will look into it.

[cwperks]

Why the addition of an extra slash when reference an ID on the API page?

[cwillum]

Apparently both ways work. I just followed the lead of the web server, which adds the forward slash when you use the link. The doc team informed me that behavior originates with the web server. In any case, I see now there's no need bothering to add it from now on. Thanks.

[cwperks]

Thinking about how to be precise and clear when communication about backend vs. way of communicating auth info (user + password combo or token) to OpenSearch.

There are 2 high-level types of IdP, Identity Providers, backends that Opensearch security plugin interfaces with. 1. Internal User database or 2. an external provider (a backend compliant with LDAP, ODIC or SAML). For client cert, jwt and proxy auth, these could be described as backend-less providers (not completely accurate, but I'll explain) in that OpenSearch does not verify the info passed in the request with a list of users and instead extracts username and backend roles from the request passed to opensearch.

1. For client cert auth, the principal of the cert is used to extract the username and backend roles. I believe username is the CN field and backend_roles are the concatenated OU fields. As long as the cert is issued by a CA in the SECURITY_SSL_HTTP_PEMTRUSTEDCAS_FILEPATH setting the cert will be trusted.
2. For JWT, a token is passed in an HTTP Authorization request header which must contain claims that match the subject_key and roles_key. The JWT must also be able to be verified with the signing_key configured in config.yml
3. For proxy auth authentication is performed at the IdP and then forwarded to opensearch. The forwarded request contains the username and backend roles in the HTTP request and OpenSearch trusts the request if its coming from the correct configured source.

The internal database is used with Basic Auth to get username and password info from an HTTP request. So in this case Basic Auth is the mechanism for transmitting auth info (username + password) in a request and the Authentication Backend would be the internal user database.

How does this sound?

The Security plugin authenticates a request against a configured backend.  Authentication providers include: Basic Auth using the internal user database, Lightweight Directory Access Protocol (LDAP)/Active Directory, JSON web tokens, SAML, or another authentication protocol.

This change you are presenting in this PR looks good to me, but stuff to keep in mind.

[cwillum]

I see the distinction. And those are good points to keep in mind. I tweaked the suggestion a little. Does it still make sense, or have I taken it off course?

The Security plugin authenticates a server request against a backend configured for an authentication provider. Some examples of authentication providers used with OpenSearch include Basic Auth (which uses the internal user database), Lightweight Directory Access Protocol (LDAP)/Active Directory, JSON web tokens, SAML, or another authentication protocol.

@cwperks Also, referring to item 1 in the flow, is it accurate to say that a user's credentials will be included in the request, no matter which authentication provider is being used? If I can say that, it will improve continuity with the statement in item 2 (continuity = better chance for making overall sense of the flow).

[cwperks]

@cwillum The tweaked suggestion sounds good to me, but instead of server request can it simply be request?

Judging by item 1, the word Credentials is used generically to mean what's needed to confer access so its accurate to say that credentials must be included on every request.

For most of the backends, Username + Password will be used for the very first request so its accurate to say the credentials are included in the request. In many cases, subsequent requests will use an access token (JWT) to prevent the need of passing username + password on every request.

In item 1:

the credentials are stored within the token itself

When I first read this it sounded like password info was embedded in the token, but that's not the case. The username (subject_key) and roles (roles_key) are embedded in the token.

[cwillum]

RE: "... but instead of server request can it simply be request?"
Definitely. I'm probably unnecessarily splitting hairs with some of these proposed changes to wording. But it's helpful to refine along the way. Thanks for your patience.

[cwperks]

Extraction of backend roles is applicable for all backends except for basic auth with the internal user database. LDAP is a bit exceptional because authz of the security plugin's config.yml needs to be configured to specify how roles should be extracted from the directory service.

For the internal user database, roles are determined by roles that the user is mapped to within opensearch. With an external IdP, a cluster admin must map backend roles to roles in OpenSearch and users will transitively get OpenSearch roles based on the roles mapping. i.e. In LDAP I may be in the My Org Developers group, and in opensearch there could be a role named developers which has a certain set of permissions. The cluster admin would need to map My Org Developers to the developers role to map the external group to the internal role within OpenSearch.

[cwillum]

@cwperks I tried to sum this up while keeping it general.
After a backend verifies the user's credentials, the plugin collects any [backend roles]({{site.url}}{{site.baseurl}}/security-plugin/access-control/index/#concepts). The authentication provider determines the way these roles are retrieved. For example, LDAP extracts backend roles from its directory service based on their mappings to roles in OpenSearch, while SAML stores the roles as attributes. When basic authentication is used, the internal user database refers to role mappings configured in OpenSearch.

[cwperks]

Sounds good to me

[cwillum]

Still waiting for a review on this from @Naarcha-AWS . Removed Tech Review tag and replaced with Doc Review. Will sound out a request on doc team Slack channel once the first is done.

[DarshitChanpura]

Should p be capital as in [Principle of least privilege]

[cwillum]

It looks like the internet generally uses lower case to name this principle. I think it should be OK.

[DarshitChanpura]

nit: indexes -> indices

And we can reword: "An alternative way to remove system indices is to delete them plugins.security.system_indices.indices\ list on each node and restart OpenSearch".

Also, I'm not sure what the original way to remove is. Should it be documented here ( maybe I'm missing something here). Thoughts?

[cwillum]

Yes. I got tripped up on this, too, and made a note of it. It comes out of the blue. If you're going to describe an alternative, you have to provide a primary way of doing something before it. Thanks. Still, I'm not sure myself what the preferred way of deleting an index is.

RE: Indices/indexes. This is probably the most hated change that developers can see in the documentation now. But the usage of "indexes" appears to have crossed the line into common usage, including AWS documentation, and the documentation is going full speed with it, with the exception of what already exists in the code.
(I even heard pundits using "indexes" on Bloomberg news the other week, finance being one of the last hold outs: FTSE, NASDAQ indices are traditional.)

I like your suggestion. When we find out the preferred way, maybe I can find a place to add both and rewrite in the singular:
"An alternative way to remove a system index is to delete it from the plugins.security.system_indices.indices\ list on each node and restart OpenSearch."
(Is the backslash needed in plugins.security.system_indices.indices\? I don't see the slash appear in the opensearch.yml.example file in Config.)

[DarshitChanpura]

Re: An alternative way... This should definitely be addressed at some point to clearly defined in the document. Not urgent for this change tho.

Re: indices/indexes I've been confused about this too, but as you mentioned in the dev world afaik it has always been indices. However, I'm completely fine with either as long as the whole documentation is aligned and points to the same usage

Re: \ yes the backslash was a typo on my end. It shouldn't contain any backslash. And, I like your suggestion to rewrite it to be singular.

[DarshitChanpura]

Thoughts on using - here instead of , surrounding which we recommend?

[cwillum]

I rewrote like so: "The approach we recommend for using the YAML files is to first configure ..."

[DarshitChanpura]

yes this is even better

[DarshitChanpura]

Thoughts on: .. the method/s to be used for authenticating users ..

[cwillum]

I think we're trying to stay away from shorthand like this in the documentation. But I would be fine with using the singular "method" alone. I don't think it confuses the meaning.

[DarshitChanpura]

Yes, singular usage sounds good and it won't divert the user interpretation of this sentence.

[DarshitChanpura]

Generic question: Changing nav_order will re-organize this document based on the value entered. Correct?

[cwillum]

Yes, it's the method for ordering topics in the TOC. This is how the TOC will look after merging this PR.

I will be breaking out two new sections for "Authentication Backends" once this is merged and the new structure is in place (Basic Authentication and JWT). But it doesn't make sense to do it in this PR, or until the new structure is in place. Once the review is complete for this, I'll get started on that.

[DarshitChanpura]

oo nice..thank you for sharing the final view. This helps a lot in visualizing the "re-organization" of the Security Plugin section

[cwillum]

Agreed. It helps.

[DarshitChanpura]

Thanks @cwillum !

[opensearch-trigger-bot]

The backport to 1.3 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-1.3 1.3
# Navigate to the new working tree
cd .worktrees/backport-1.3
# Create a new branch
git switch --create backport/backport-2212-to-1.3
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 d04e88b496485474fc3d89bd42ea36e834d6a64c
# Push it to GitHub
git push --set-upstream origin backport/backport-2212-to-1.3
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-1.3

Then, create a pull request where the base branch is 1.3 and the compare/head branch is backport/backport-2212-to-1.3.

[opensearch-trigger-bot]

The backport to 2.0 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.0 2.0
# Navigate to the new working tree
cd .worktrees/backport-2.0
# Create a new branch
git switch --create backport/backport-2212-to-2.0
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 d04e88b496485474fc3d89bd42ea36e834d6a64c
# Push it to GitHub
git push --set-upstream origin backport/backport-2212-to-2.0
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.0

Then, create a pull request where the base branch is 2.0 and the compare/head branch is backport/backport-2212-to-2.0.

[opensearch-trigger-bot]

The backport to 2.2 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.2 2.2
# Navigate to the new working tree
cd .worktrees/backport-2.2
# Create a new branch
git switch --create backport/backport-2212-to-2.2
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 d04e88b496485474fc3d89bd42ea36e834d6a64c
# Push it to GitHub
git push --set-upstream origin backport/backport-2212-to-2.2
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.2

Then, create a pull request where the base branch is 2.2 and the compare/head branch is backport/backport-2212-to-2.2.

[opensearch-trigger-bot]

The backport to 2.1 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.1 2.1
# Navigate to the new working tree
cd .worktrees/backport-2.1
# Create a new branch
git switch --create backport/backport-2212-to-2.1
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 d04e88b496485474fc3d89bd42ea36e834d6a64c
# Push it to GitHub
git push --set-upstream origin backport/backport-2212-to-2.1
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.1

Then, create a pull request where the base branch is 2.1 and the compare/head branch is backport/backport-2212-to-2.1.

[opensearch-trigger-bot]

The backport to 2.3 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.3 2.3
# Navigate to the new working tree
cd .worktrees/backport-2.3
# Create a new branch
git switch --create backport/backport-2212-to-2.3
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 d04e88b496485474fc3d89bd42ea36e834d6a64c
# Push it to GitHub
git push --set-upstream origin backport/backport-2212-to-2.3
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.3

Then, create a pull request where the base branch is 2.3 and the compare/head branch is backport/backport-2212-to-2.3.