diff --git a/x-pack/docs/en/security/limitations.asciidoc b/x-pack/docs/en/security/limitations.asciidoc deleted file mode 100644 index fb8b826d5dd58..0000000000000 --- a/x-pack/docs/en/security/limitations.asciidoc +++ /dev/null @@ -1,87 +0,0 @@ -[role="xpack"] -[[security-limitations]] -== Security Limitations - -[float] -=== Plugins - -Elasticsearch's plugin infrastructure is extremely flexible in terms of what can -be extended. While it opens up Elasticsearch to a wide variety of (often custom) -additional functionality, when it comes to security, this high extensibility level -comes at a cost. We have no control over the third-party plugins' code (open -source or not) and therefore we cannot guarantee their compliance with {security}. -For this reason, third-party plugins are not officially supported on clusters -with {security} enabled. - -[float] -=== Changes in Index Wildcard Behavior - -Elasticsearch clusters with {security} enabled apply the `/_all` wildcard, and -all other wildcards, to the indices that the current user has privileges for, not -the set of all indices on the cluster. - -[float] -=== Multi Document APIs - -Multi get and multi term vectors API throw IndexNotFoundException when trying to access non existing indices that the user is -not authorized for. By doing that they leak information regarding the fact that the index doesn't exist, while the user is not -authorized to know anything about those indices. - -[float] -=== Filtered Index Aliases - -Aliases containing filters are not a secure way to restrict access to individual -documents, due to the limitations described in <>. -{security} provides a secure way to restrict access to documents through the -<> feature. - -[float] -=== Field and Document Level Security Limitations - -When a user's role enables document or field level security for an index: - -* The user cannot perform write operations: -** The update API isn't supported. -** Update requests included in bulk requests aren't supported. -* The request cache is disabled for search requests. - -When a user's role enables document level security for an index: - -* Document level security isn't applied for APIs that aren't document based. - An example is the field stats API. -* Document level security doesn't affect global index statistics that relevancy - scoring uses. So this means that scores are computed without taking the role - query into account. Note that documents not matching with the role query are - never returned. -* The `has_child` and `has_parent` queries aren't supported as query in the - role definition. The `has_child` and `has_parent` queries can be used in the - search API with document level security enabled. -* Any query that makes remote calls to fetch data to query by isn't supported. - The following queries aren't supported: -** The `terms` query with terms lookup isn't supported. -** The `geo_shape` query with indexed shapes isn't supported. -** The `percolate` query isn't supported. -* If suggesters are specified and document level security is enabled then - the specified suggesters are ignored. -* A search request cannot be profiled if document level security is enabled. - -[float] -[[alias-limitations]] -=== Index and Field Names Can Be Leaked When Using Aliases - -Calling certain Elasticsearch APIs on an alias can potentially leak information -about indices that the user isn't authorized to access. For example, when you get -the mappings for an alias with the `_mapping` API, the response includes the -index name and mappings for each index that the alias applies to. - -Until this limitation is addressed, avoid index and field names that contain -confidential or sensitive information. - -[float] -=== LDAP Realm - -The <> does not currently support the discovery of nested -LDAP Groups. For example, if a user is a member of `group_1` and `group_1` is a -member of `group_2`, only `group_1` will be discovered. However, the -<> *does* support transitive -group membership. diff --git a/x-pack/docs/en/security/troubleshooting.asciidoc b/x-pack/docs/en/security/troubleshooting.asciidoc deleted file mode 100644 index 72a05ada29958..0000000000000 --- a/x-pack/docs/en/security/troubleshooting.asciidoc +++ /dev/null @@ -1,490 +0,0 @@ -[role="xpack"] -[[security-troubleshooting]] -== {security} Troubleshooting -++++ -{security} -++++ - -Use the information in this section to troubleshoot common problems and find -answers for frequently asked questions. - -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> - - -To get help, see <>. - -[[security-trb-settings]] -=== Some settings are not returned via the nodes settings API - -*Symptoms:* - -* When you use the {ref}/cluster-nodes-info.html[nodes info API] to retrieve -settings for a node, some information is missing. - -*Resolution:* - -This is intentional. Some of the settings are considered to be highly -sensitive: all `ssl` settings, ldap `bind_dn`, and `bind_password`. -For this reason, we filter these settings and do not expose them via -the nodes info API rest endpoint. You can also define additional -sensitive settings that should be hidden using the -`xpack.security.hide_settings` setting. For example, this snippet -hides the `url` settings of the `ldap1` realm and all settings of the -`ad1` realm. - -[source, yaml] ------------------------------------------- -xpack.security.hide_settings: xpack.security.authc.realms.ldap1.url, -xpack.security.authc.realms.ad1.* ------------------------------------------- - -[[security-trb-roles]] -=== Authorization exceptions - -*Symptoms:* - -* I configured the appropriate roles and the users, but I still get an -authorization exception. -* I can authenticate to LDAP, but I still get an authorization exception. - - -*Resolution:* - -. Verify that the role names associated with the users match the roles defined -in the `roles.yml` file. You can use the `elasticsearch-users` tool to list all -the users. Any unknown roles are marked with `*`. -+ --- -[source, shell] ------------------------------------------- -bin/elasticsearch-users list -rdeniro : admin -alpacino : power_user -jacknich : monitoring,unknown_role* <1> ------------------------------------------- -<1> `unknown_role` was not found in `roles.yml` - -For more information about this command, see the -{ref}/users-command.html[`elasticsearch-users` command]. --- - -. If you are authenticating to LDAP, a number of configuration options can cause -this error. -+ --- -|====================== -|_group identification_ | - -Groups are located by either an LDAP search or by the "memberOf" attribute on -the user. Also, If subtree search is turned off, it will search only one -level deep. See the <> for all the options. -There are many options here and sticking to the defaults will not work for all -scenarios. - -| _group to role mapping_| - -Either the `role_mapping.yml` file or the location for this file could be -misconfigured. See <> for more. - -|_role definition_| - -The role definition might be missing or invalid. - -|====================== - -To help track down these possibilities, add the following lines to the end of -the `log4j2.properties` configuration file in the `ES_PATH_CONF`: - -[source,properties] ----------------- -logger.authc.name = org.elasticsearch.xpack.security.authc -logger.authc.level = DEBUG ----------------- - -A successful authentication should produce debug statements that list groups and -role mappings. --- - -[[security-trb-extraargs]] -=== Users command fails due to extra arguments - -*Symptoms:* - -* The `elasticsearch-users` command fails with the following message: -`ERROR: extra arguments [...] were provided`. - -*Resolution:* - -This error occurs when the `elasticsearch-users` tool is parsing the input and -finds unexpected arguments. This can happen when there are special characters -used in some of the arguments. For example, on Windows systems the `,` character -is considered a parameter separator; in other words `-r role1,role2` is -translated to `-r role1 role2` and the `elasticsearch-users` tool only -recognizes `role1` as an expected parameter. The solution here is to quote the -parameter: `-r "role1,role2"`. - -For more information about this command, see -{ref}/users-command.html[`elasticsearch-users` command]. - -[[trouble-shoot-active-directory]] -=== Users are frequently locked out of Active Directory - -*Symptoms:* - -* Certain users are being frequently locked out of Active Directory. - -*Resolution:* - -Check your realm configuration; realms are checked serially, one after another. -If your Active Directory realm is being checked before other realms and there -are usernames that appear in both Active Directory and another realm, a valid -login for one realm might be causing failed login attempts in another realm. - -For example, if `UserA` exists in both Active Directory and a file realm, and -the Active Directory realm is checked first and file is checked second, an -attempt to authenticate as `UserA` in the file realm would first attempt to -authenticate against Active Directory and fail, before successfully -authenticating against the `file` realm. Because authentication is verified on -each request, the Active Directory realm would be checked - and fail - on each -request for `UserA` in the `file` realm. In this case, while the authentication -request completed successfully, the account on Active Directory would have -received several failed login attempts, and that account might become -temporarily locked out. Plan the order of your realms accordingly. - -Also note that it is not typically necessary to define multiple Active Directory -realms to handle domain controller failures. When using Microsoft DNS, the DNS -entry for the domain should always point to an available domain controller. - - -[[trb-security-maccurl]] -=== Certificate verification fails for curl on Mac - -*Symptoms:* - -* `curl` on the Mac returns a certificate verification error even when the -`--cacert` option is used. - - -*Resolution:* - -Apple's integration of `curl` with their keychain technology disables the -`--cacert` option. -See http://curl.haxx.se/mail/archive-2013-10/0036.html for more information. - -You can use another tool, such as `wget`, to test certificates. Alternately, you -can add the certificate for the signing certificate authority MacOS system -keychain, using a procedure similar to the one detailed at the -http://support.apple.com/kb/PH14003[Apple knowledge base]. Be sure to add the -signing CA's certificate and not the server's certificate. - - -[[trb-security-sslhandshake]] -=== SSLHandshakeException causes connections to fail - -*Symptoms:* - -* A `SSLHandshakeException` causes a connection to a node to fail and indicates -that there is a configuration issue. Some of the common exceptions are shown -below with tips on how to resolve these issues. - - -*Resolution:* - -`java.security.cert.CertificateException: No name matching node01.example.com found`:: -+ --- -Indicates that a client connection was made to `node01.example.com` but the -certificate returned did not contain the name `node01.example.com`. In most -cases, the issue can be resolved by ensuring the name is specified during -certificate creation. For more information, see <>. Another scenario is -when the environment does not wish to use DNS names in certificates at all. In -this scenario, all settings in `elasticsearch.yml` should only use IP addresses -including the `network.publish_host` setting. --- - -`java.security.cert.CertificateException: No subject alternative names present`:: -+ --- -Indicates that a client connection was made to an IP address but the returned -certificate did not contain any `SubjectAlternativeName` entries. IP addresses -are only used for hostname verification if they are specified as a -`SubjectAlternativeName` during certificate creation. If the intent was to use -IP addresses for hostname verification, then the certificate will need to be -regenerated with the appropriate IP address. See <>. --- - -`javax.net.ssl.SSLHandshakeException: null cert chain` and `javax.net.ssl.SSLException: Received fatal alert: bad_certificate`:: -+ --- -The `SSLHandshakeException` indicates that a self-signed certificate was -returned by the client that is not trusted as it cannot be found in the -`truststore` or `keystore`. This `SSLException` is seen on the client side of -the connection. --- - -`sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target` and `javax.net.ssl.SSLException: Received fatal alert: certificate_unknown`:: -+ --- -This `SunCertPathBuilderException` indicates that a certificate was returned -during the handshake that is not trusted. This message is seen on the client -side of the connection. The `SSLException` is seen on the server side of the -connection. The CA certificate that signed the returned certificate was not -found in the `keystore` or `truststore` and needs to be added to trust this -certificate. --- - -[[trb-security-ssl]] -=== Common SSL/TLS exceptions - -*Symptoms:* - -* You might see some exceptions related to SSL/TLS in your logs. Some of the -common exceptions are shown below with tips on how to resolve these issues. + - - - -*Resolution:* - -`WARN: received plaintext http traffic on a https channel, closing connection`:: -+ --- -Indicates that there was an incoming plaintext http request. This typically -occurs when an external applications attempts to make an unencrypted call to the -REST interface. Please ensure that all applications are using `https` when -calling the REST interface with SSL enabled. --- - -`org.elasticsearch.common.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record:`:: -+ --- -Indicates that there was incoming plaintext traffic on an SSL connection. This -typically occurs when a node is not configured to use encrypted communication -and tries to connect to nodes that are using encrypted communication. Please -verify that all nodes are using the same setting for -`xpack.security.transport.ssl.enabled`. - -For more information about this setting, see -{ref}/security-settings.html[Security Settings in {es}]. --- - -`java.io.StreamCorruptedException: invalid internal transport message format, got`:: -+ --- -Indicates an issue with data received on the transport interface in an unknown -format. This can happen when a node with encrypted communication enabled -connects to a node that has encrypted communication disabled. Please verify that -all nodes are using the same setting for `xpack.security.transport.ssl.enabled`. - -For more information about this setting, see -{ref}/security-settings.html[Security Settings in {es}]. --- - -`java.lang.IllegalArgumentException: empty text`:: -+ --- -This exception is typically seen when a `https` request is made to a node that -is not using `https`. If `https` is desired, please ensure the following setting -is in `elasticsearch.yml`: - -[source,yaml] ----------------- -xpack.security.http.ssl.enabled: true ----------------- - -For more information about this setting, see -{ref}/security-settings.html[Security Settings in {es}]. --- - -`ERROR: unsupported ciphers [...] were requested but cannot be used in this JVM`:: -+ --- -This error occurs when a SSL/TLS cipher suite is specified that cannot supported -by the JVM that {es} is running in. Security tries to use the specified cipher -suites that are supported by this JVM. This error can occur when using the -Security defaults as some distributions of OpenJDK do not enable the PKCS11 -provider by default. In this case, we recommend consulting your JVM -documentation for details on how to enable the PKCS11 provider. - -Another common source of this error is requesting cipher suites that use -encrypting with a key length greater than 128 bits when running on an Oracle JDK. -In this case, you must install the -<>. --- - -[[trb-security-kerberos]] -=== Common Kerberos exceptions - -*Symptoms:* - -* User authentication fails due to either GSS negotiation failure -or a service login failure (either on the server or in the {es} http client). -Some of the common exceptions are listed below with some tips to help resolve -them. - -*Resolution:* - -`Failure unspecified at GSS-API level (Mechanism level: Checksum failed)`:: -+ --- - -When you see this error message on the HTTP client side, then it may be -related to an incorrect password. - -When you see this error message in the {es} server logs, then it may be -related to the {es} service keytab. The keytab file is present but it failed -to log in as the user. Please check the keytab expiry. Also check whether the -keytab contain up-to-date credentials; if not, replace them. - -You can use tools like `klist` or `ktab` to list principals inside -the keytab and validate them. You can use `kinit` to see if you can acquire -initial tickets using the keytab. Please check the tools and their documentation -in your Kerberos environment. - -Kerberos depends on proper hostname resolution, so please check your DNS infrastructure. -Incorrect DNS setup, DNS SRV records or configuration for KDC servers in `krb5.conf` -can cause problems with hostname resolution. - --- - -`Failure unspecified at GSS-API level (Mechanism level: Request is a replay (34))`:: - -`Failure unspecified at GSS-API level (Mechanism level: Clock skew too great (37))`:: -+ --- - -To prevent replay attacks, Kerberos V5 sets a maximum tolerance for computer -clock synchronization and it is typically 5 minutes. Please check whether -the time on the machines within the domain is in sync. - --- - -As Kerberos logs are often cryptic in nature and many things can go wrong -as it depends on external services like DNS and NTP. You might -have to enable additional debug logs to determine the root cause of the issue. - -{es} uses a JAAS (Java Authentication and Authorization Service) Kerberos login -module to provide Kerberos support. To enable debug logs on {es} for the login -module use following Kerberos realm setting: -[source,yaml] ----------------- -xpack.security.authc.realms..krb.debug: true ----------------- - -For detailed information, see {ref}/security-settings.html#ref-kerberos-settings[Kerberos realm settings]. - -Sometimes you may need to go deeper to understand the problem during SPNEGO -GSS context negotiation or look at the Kerberos message exchange. To enable -Kerberos/SPNEGO debug logging on JVM, add following JVM system properties: - -`-Dsun.security.krb5.debug=true` - -`-Dsun.security.spnego.debug=true` - -For more information about JVM system properties, see {ref}/jvm-options.html[configuring JVM options]. - -[[trb-security-internalserver]] -=== Internal Server Error in Kibana - -*Symptoms:* - -* In 5.1.1, an `UnhandledPromiseRejectionWarning` occurs and {kib} displays an -Internal Server Error. -//TBD: Is the same true for later releases? - -*Resolution:* - -If the Security plugin is enabled in {es} but disabled in {kib}, you must -still set `elasticsearch.username` and `elasticsearch.password` in `kibana.yml`. -Otherwise, {kib} cannot connect to {es}. - - -[[trb-security-setup]] -=== Setup-passwords command fails due to connection failure - -The {ref}/setup-passwords.html[elasticsearch-setup-passwords command] sets -passwords for the built-in users by sending user management API requests. If -your cluster uses SSL/TLS for the HTTP (REST) interface, the command attempts to -establish a connection with the HTTPS protocol. If the connection attempt fails, -the command fails. - -*Symptoms:* - -. {es} is running HTTPS, but the command fails to detect it and returns the -following errors: -+ --- -[source, shell] ------------------------------------------- -Cannot connect to elasticsearch node. -java.net.SocketException: Unexpected end of file from server -... -ERROR: Failed to connect to elasticsearch at -http://127.0.0.1:9200/_xpack/security/_authenticate?pretty. -Is the URL correct and elasticsearch running? ------------------------------------------- --- - -. SSL/TLS is configured, but trust cannot be established. The command returns -the following errors: -+ --- -[source, shell] ------------------------------------------- -SSL connection to -https://127.0.0.1:9200/_xpack/security/_authenticate?pretty -failed: sun.security.validator.ValidatorException: -PKIX path building failed: -sun.security.provider.certpath.SunCertPathBuilderException: -unable to find valid certification path to requested target -Please check the elasticsearch SSL settings under -xpack.security.http.ssl. -... -ERROR: Failed to establish SSL connection to elasticsearch at -https://127.0.0.1:9200/_xpack/security/_authenticate?pretty. ------------------------------------------- --- - -. The command fails because hostname verification fails, which results in the -following errors: -+ --- -[source, shell] ------------------------------------------- -SSL connection to -https://idp.localhost.test:9200/_xpack/security/_authenticate?pretty -failed: java.security.cert.CertificateException: -No subject alternative DNS name matching -elasticsearch.example.com found. -Please check the elasticsearch SSL settings under -xpack.security.http.ssl. -... -ERROR: Failed to establish SSL connection to elasticsearch at -https://elasticsearch.example.com:9200/_xpack/security/_authenticate?pretty. ------------------------------------------- --- - -*Resolution:* - -. If your cluster uses TLS/SSL for the HTTP interface but the -`elasticsearch-setup-passwords` command attempts to establish a non-secure -connection, use the `--url` command option to explicitly specify an HTTPS URL. -Alternatively, set the `xpack.security.http.ssl.enabled` setting to `true`. - -. If the command does not trust the {es} server, verify that you configured the -`xpack.security.http.ssl.certificate_authorities` setting or the -`xpack.security.http.ssl.truststore.path` setting. - -. If hostname verification fails, you can disable this verification by setting -`xpack.security.http.ssl.verification_mode` to `certificate`. - -For more information about these settings, see -{ref}/security-settings.html[Security Settings in {es}]. diff --git a/x-pack/docs/en/watcher/images/watcher-ui-edit-watch.png b/x-pack/docs/en/watcher/images/watcher-ui-edit-watch.png deleted file mode 100644 index f6a3ab4354a21..0000000000000 Binary files a/x-pack/docs/en/watcher/images/watcher-ui-edit-watch.png and /dev/null differ diff --git a/x-pack/docs/en/watcher/limitations.asciidoc b/x-pack/docs/en/watcher/limitations.asciidoc deleted file mode 100644 index 9ae7273de71db..0000000000000 --- a/x-pack/docs/en/watcher/limitations.asciidoc +++ /dev/null @@ -1,28 +0,0 @@ -[[watcher-limitations]] -== Watcher Limitations - -[float] -=== Watches Are Not Updated When File Based Scripts Change - -When you refer to a file script in a watch, the watch itself is not updated -if you change the script on the filesystem. - -Currently, the only way to reload a file script in a watch is to delete -the watch and recreate it. - -[float] -=== Watcher UI - -When you create a new watch or edit an existing watch, if you navigate away -from the page without saving your changes they will be lost without warning. -Make sure to save your changes before leaving the page. - -image::watcher-ui-edit-watch.png[] - -[float] -=== Security Integration - -When {security} is enabled, a watch stores information about what the user who -stored the watch is allowed to execute **at that time**. This means, if those -permissions change over time, the watch will still be able to execute with the -permissions that existed when the watch was created. diff --git a/x-pack/docs/en/watcher/troubleshooting.asciidoc b/x-pack/docs/en/watcher/troubleshooting.asciidoc deleted file mode 100644 index 20d599f8f5215..0000000000000 --- a/x-pack/docs/en/watcher/troubleshooting.asciidoc +++ /dev/null @@ -1,63 +0,0 @@ -[[watcher-troubleshooting]] -== {xpack} {watcher} Troubleshooting -++++ -{xpack} {watcher} -++++ - -[float] -=== Dynamic Mapping Error When Trying to Add a Watch - -If you get the _Dynamic Mapping is Disabled_ error when you try to add a watch, -verify that the index mappings for the `.watches` index are available. You can -do that by submitting the following request: - -[source,js] --------------------------------------------------- -GET .watches/_mapping --------------------------------------------------- -// CONSOLE -// TEST[setup:my_active_watch] - -If the index mappings are missing, follow these steps to restore the correct -mappings: - -. Stop the Elasticsearch node. -. Add `xpack.watcher.index.rest.direct_access : true` to `elasticsearch.yml`. -. Restart the Elasticsearch node. -. Delete the `.watches` index: -+ -[source,js] --------------------------------------------------- -DELETE .watches --------------------------------------------------- -// CONSOLE -// TEST[skip:index deletion] -+ -. Disable direct access to the `.watches` index: -.. Stop the Elasticsearch node. -.. Remove `xpack.watcher.index.rest.direct_access : true` from `elasticsearch.yml`. -.. Restart the Elasticsearch node. - -[float] -=== Unable to Send Email - -If you get an authentication error indicating that you need to continue the -sign-in process from a web browser when Watcher attempts to send email, you need -to configure Gmail to -https://support.google.com/accounts/answer/6010255?hl=en[Allow Less Secure Apps to access your account]. - -If you have two-step verification enabled for your email account, you must -generate and use an App Specific password to send email from {watcher}. For more -information, see: - -- Gmail: https://support.google.com/accounts/answer/185833?hl=en[Sign in using App Passwords] -- Outlook.com: http://windows.microsoft.com/en-us/windows/app-passwords-two-step-verification[App passwords and two-step verification] - -[float] -=== {watcher} Not Responsive - -Keep in mind that there's no built-in validation of scripts that you add to a -watch. Buggy or deliberately malicious scripts can negatively impact {watcher} -performance. For example, if you add multiple watches with buggy script -conditions in a short period of time, {watcher} might be temporarily unable to -process watches until the bad watches time out.