Merge branch 'elastic:main' into fix-ltr-useless-field-value-extractor

docs/changelog/110236.yaml

@@ -0,0 +1,21 @@
+pr: 110236
+summary: '`ParseHeapRatioOrDeprecatedByteSizeValue` for `indices.breaker.total.limit`'
+area: Infra/Settings
+type: deprecation
+issues: []
+deprecation:
+  title: 'Deprecate absolute size values for `indices.breaker.total.limit` setting'
+  area: Cluster and node setting
+  details: Previously, the value of `indices.breaker.total.limit` could be specified as
+    an absolute size in bytes. This setting controls the overal amount of
+    memory the server is allowed to use before taking remedial actions. Setting
+    this to a specific number of bytes led to strange behaviour when the node
+    maximum heap size changed because the circut breaker limit would remain
+    unchanged. This would either leave the value too low, causing part of the
+    heap to remain unused; or it would leave the value too high, causing the
+    circuit breaker to be ineffective at preventing OOM errors.  The only
+    reasonable behaviour for this setting is that it scales with the size of
+    the heap, and so absolute byte limits are now deprecated.
+  impact: Users must change their configuration to specify a percentage instead of
+    an absolute number of bytes for `indices.breaker.total.limit`, or else
+    accept the default, which is already specified as a percentage.

docs/reference/rest-api/security.asciidoc

@@ -45,6 +45,7 @@ Use the following APIs to add, remove, update, and retrieve role mappings:
 Use the following APIs to add, remove, update, and retrieve roles in the native realm:
 
 * <<security-api-put-role,Create or update roles>>
+* <<security-api-bulk-put-role, Bulk create or update roles>>
 * <<security-api-clear-role-cache,Clear roles cache>>
 * <<security-api-delete-role,Delete roles>>
 * <<security-api-get-role,Get roles>>
@@ -171,6 +172,7 @@ include::security/create-api-keys.asciidoc[]
 include::security/put-app-privileges.asciidoc[]
 include::security/create-role-mappings.asciidoc[]
 include::security/create-roles.asciidoc[]
+include::security/bulk-create-roles.asciidoc[]
 include::security/create-users.asciidoc[]
 include::security/create-service-token.asciidoc[]
 include::security/delegate-pki-authentication.asciidoc[]

docs/reference/rest-api/security/bulk-create-roles.asciidoc

@@ -0,0 +1,328 @@
+[role="xpack"]
+[[security-api-bulk-put-role]]
+=== Bulk create or update roles API
+preview::[]
+++++
+<titleabbrev>Bulk create or update roles API</titleabbrev>
+++++
+
+Bulk adds and updates roles in the native realm.
+
+[[security-api-bulk-put-role-request]]
+==== {api-request-title}
+
+`POST /_security/role/` +
+
+[[security-api-bulk-put-role-prereqs]]
+==== {api-prereq-title}
+
+* To use this API, you must have at least the `manage_security` cluster
+privilege.
+
+[[security-api-bulk-put-role-desc]]
+==== {api-description-title}
+
+The role management APIs are generally the preferred way to manage roles, rather than using
+<<roles-management-file,file-based role management>>. The bulk create
+or update roles API cannot update roles that are defined in roles files.
+
+[[security-api-bulk-put-role-path-params]]
+==== {api-path-parms-title}
+
+`refresh`::
+Optional setting of the {ref}/docs-refresh.html[refresh policy] for the write request. Defaults to Immediate.
+
+[[security-api-bulk-put-role-request-body]]
+==== {api-request-body-title}
+
+The following parameters can be specified in the body of a POST request
+and pertain to adding a set of roles:
+
+`roles`::
+(object) The roles to add as a role name to role map.
+
+====
+`<role_name>` (required):: (string) The role name.
+`applications`:: (list) A list of application privilege entries.
+`application` (required)::: (string) The name of the application to which this entry applies.
+`privileges`::: (list) A list of strings, where each element is the name of an application
+privilege or action.
+`resources`::: (list) A list resources to which the privileges are applied.
+
+`cluster`:: (list) A list of cluster privileges. These privileges define the
+cluster level actions that users with this role are able to execute.
+
+`global`:: (object) An object defining global privileges. A global privilege is
+a form of cluster privilege that is request-aware. Support for global privileges
+is currently limited to the management of application privileges.
+
+`indices`:: (list) A list of indices permissions entries.
+`field_security`::: (object) The document fields that the owners of the role have
+read access to. For more information, see
+<<field-and-document-access-control>>.
+`names` (required)::: (list) A list of indices (or index name patterns) to which the
+permissions in this entry apply.
+`privileges`(required)::: (list) The index level privileges that the owners of the role
+have on the specified indices.
+`query`::: A search query that defines the documents the owners of the role have
+read access to. A document within the specified indices must match this query in
+order for it to be accessible by the owners of the role.
+
+`metadata`:: (object) Optional meta-data. Within the `metadata` object, keys
+that begin with `_` are reserved for system usage.
+
+`run_as`:: (list) A list of users that the owners of this role can impersonate.
+For more information, see
+<<run-as-privilege>>.
+
+`remote_indices`:: beta:[] (list) A list of remote indices permissions entries.
++
+--
+NOTE: Remote indices are effective for <<remote-clusters-api-key,remote clusters configured with the API key based model>>.
+They have no effect for remote clusters configured with the <<remote-clusters-cert,certificate based model>>.
+--
+`clusters` (required)::: (list) A list of cluster aliases to which the permissions
+in this entry apply.
+`field_security`::: (object) The document fields that the owners of the role have
+read access to. For more information, see
+<<field-and-document-access-control>>.
+`names` (required)::: (list) A list of indices (or index name patterns) on the remote clusters
+(specified with `clusters`) to which the permissions in this entry apply.
+`privileges`(required)::: (list) The index level privileges that the owners of the role
+have on the specified indices.
+`query`::: A search query that defines the documents the owners of the role have
+read access to. A document within the specified indices must match this query in
+order for it to be accessible by the owners of the role.
+
+For more information, see <<defining-roles>>.
+====
+
+[[security-bulk-api-put-role-example]]
+==== {api-examples-title}
+
+The following example adds the roles called `my_admin_role` and `my_user_role`:
+
+[source,console]
+--------------------------------------------------
+POST /_security/role
+{
+    "roles": {
+        "my_admin_role": {
+            "cluster": [
+                "all"
+            ],
+            "indices": [
+                {
+                    "names": [
+                        "index1",
+                        "index2"
+                    ],
+                    "privileges": [
+                        "all"
+                    ],
+                    "field_security": {
+                        "grant": [
+                            "title",
+                            "body"
+                        ]
+                    },
+                    "query": "{\"match\": {\"title\": \"foo\"}}"
+                }
+            ],
+            "applications": [
+                {
+                    "application": "myapp",
+                    "privileges": [
+                        "admin",
+                        "read"
+                    ],
+                    "resources": [
+                        "*"
+                    ]
+                }
+            ],
+            "run_as": [
+                "other_user"
+            ],
+            "metadata": {
+                "version": 1
+            }
+        },
+        "my_user_role": {
+            "cluster": [
+                "all"
+            ],
+            "indices": [
+                {
+                    "names": [
+                        "index1"
+                    ],
+                    "privileges": [
+                        "read"
+                    ],
+                    "field_security": {
+                        "grant": [
+                            "title",
+                            "body"
+                        ]
+                    },
+                    "query": "{\"match\": {\"title\": \"foo\"}}"
+                }
+            ],
+            "applications": [
+                {
+                    "application": "myapp",
+                    "privileges": [
+                        "admin",
+                        "read"
+                    ],
+                    "resources": [
+                        "*"
+                    ]
+                }
+            ],
+            "run_as": [
+                "other_user"
+            ],
+            "metadata": {
+                "version": 1
+            }
+        }
+    }
+}
+--------------------------------------------------
+
+A successful call returns a JSON structure that shows whether the role has been
+created, updated, or had no changes made.
+
+[source,console-result]
+--------------------------------------------------
+{
+    "created": [ <1>
+        "my_admin_role", <2>
+        "my_user_role"
+    ]
+}
+--------------------------------------------------
+
+<1> Result type, one of `created`, `updated`, `noop`, `errors`.
+<2> A list of the roles that were created.
+
+Because errors are handled individually for each role create or update, the API allows partial success.
+
+The following query would throw an error for `my_admin_role` because the privilege `bad_cluster_privilege`
+doesn't exist, but would be successful for the `my_user_role`.
+
+[source,console]
+--------------------------------------------------
+POST /_security/role
+{
+    "roles": {
+        "my_admin_role": {
+            "cluster": [
+                "bad_cluster_privilege"
+            ],
+            "indices": [
+                {
+                    "names": [
+                        "index1",
+                        "index2"
+                    ],
+                    "privileges": ["all"],
+                    "field_security": {
+                        "grant": [
+                            "title",
+                            "body"
+                        ]
+                    },
+                    "query": "{\"match\": {\"title\": \"foo\"}}"
+                }
+            ],
+            "applications": [
+                {
+                    "application": "myapp",
+                    "privileges": [
+                        "admin",
+                        "read"
+                    ],
+                    "resources": [
+                        "*"
+                    ]
+                }
+            ],
+            "run_as": [
+                "other_user"
+            ],
+            "metadata": {
+                "version": 1
+            }
+        },
+        "my_user_role": {
+            "cluster": [
+                "all"
+            ],
+            "indices": [
+                {
+                    "names": [
+                        "index1"
+                    ],
+                    "privileges": [
+                        "read"
+                    ],
+                    "field_security": {
+                        "grant": [
+                            "title",
+                            "body"
+                        ]
+                    },
+                    "query": "{\"match\": {\"title\": \"foo\"}}"
+                }
+            ],
+            "applications": [
+                {
+                    "application": "myapp",
+                    "privileges": [
+                        "admin",
+                        "read"
+                    ],
+                    "resources": [
+                        "*"
+                    ]
+                }
+            ],
+            "run_as": [
+                "other_user"
+            ],
+            "metadata": {
+                "version": 1
+            }
+        }
+    }
+}
+--------------------------------------------------
+
+The result would then have the `errors` field set to `true` and hold the error for the `my_admin_role` update.
+
+
+[source,console-result]
+--------------------------------------------------
+{
+    "created": [
+        "my_user_role" <1>
+    ],
+    "errors": { <2>
+        "count": 1, <3>
+        "details": {
+            "my_admin_role": { <4>
+                "type": "action_request_validation_exception",
+                "reason": "Validation Failed: 1: unknown cluster privilege [bad_cluster_privilege]. a privilege must be either one of the predefined cluster privilege names [manage_own_api_key,none,cancel_task,cross_cluster_replication,cross_cluster_search,delegate_pki,grant_api_key,manage_autoscaling,manage_index_templates,manage_logstash_pipelines,manage_oidc,manage_saml,manage_search_application,manage_search_query_rules,manage_search_synonyms,manage_service_account,manage_token,manage_user_profile,monitor_connector,monitor_data_stream_global_retention,monitor_enrich,monitor_inference,monitor_ml,monitor_rollup,monitor_snapshot,monitor_text_structure,monitor_watcher,post_behavioral_analytics_event,read_ccr,read_connector_secrets,read_fleet_secrets,read_ilm,read_pipeline,read_security,read_slm,transport_client,write_connector_secrets,write_fleet_secrets,create_snapshot,manage_behavioral_analytics,manage_ccr,manage_connector,manage_data_stream_global_retention,manage_enrich,manage_ilm,manage_inference,manage_ml,manage_rollup,manage_slm,manage_watcher,monitor_data_frame_transforms,monitor_transform,manage_api_key,manage_ingest_pipelines,manage_pipeline,manage_data_frame_transforms,manage_transform,manage_security,monitor,manage,all] or a pattern over one of the available cluster actions;"
+            }
+        }
+    }
+}
+--------------------------------------------------
+
+<1> The successfully created role.
+<2> The errors encountered.
+<3> The number of put role requests that resulted in an error.
+<4> The error keyed by role name.