From 408db3b041e3fd17124d580c2ff51847ac67c9a8 Mon Sep 17 00:00:00 2001 From: Adam Locke Date: Thu, 29 Apr 2021 16:26:14 -0400 Subject: [PATCH] [DOCS] [7.x] removing old security files no longer used (#72496) (#72502) --- .../get-started-builtin-users.asciidoc | 33 -- .../get-started-enable-security.asciidoc | 35 -- .../get-started-kibana-users.asciidoc | 62 --- .../en/security/get-started-security.asciidoc | 377 ------------------ .../node-certificates.asciidoc | 138 ------- .../securing-elasticsearch.asciidoc | 42 -- .../tls-transport.asciidoc | 109 ----- .../tutorial-tls-addnodes.asciidoc | 188 --------- .../tutorial-tls-certificates.asciidoc | 77 ---- .../tutorial-tls-internode.asciidoc | 177 -------- .../tutorial-tls-intro.asciidoc | 47 --- 11 files changed, 1285 deletions(-) delete mode 100644 x-pack/docs/en/security/get-started-builtin-users.asciidoc delete mode 100644 x-pack/docs/en/security/get-started-enable-security.asciidoc delete mode 100644 x-pack/docs/en/security/get-started-kibana-users.asciidoc delete mode 100644 x-pack/docs/en/security/get-started-security.asciidoc delete mode 100644 x-pack/docs/en/security/securing-communications/node-certificates.asciidoc delete mode 100644 x-pack/docs/en/security/securing-communications/securing-elasticsearch.asciidoc delete mode 100644 x-pack/docs/en/security/securing-communications/tls-transport.asciidoc delete mode 100644 x-pack/docs/en/security/securing-communications/tutorial-tls-addnodes.asciidoc delete mode 100644 x-pack/docs/en/security/securing-communications/tutorial-tls-certificates.asciidoc delete mode 100644 x-pack/docs/en/security/securing-communications/tutorial-tls-internode.asciidoc delete mode 100644 x-pack/docs/en/security/securing-communications/tutorial-tls-intro.asciidoc diff --git a/x-pack/docs/en/security/get-started-builtin-users.asciidoc b/x-pack/docs/en/security/get-started-builtin-users.asciidoc deleted file mode 100644 index b774f47d1c0d4..0000000000000 --- a/x-pack/docs/en/security/get-started-builtin-users.asciidoc +++ /dev/null @@ -1,33 +0,0 @@ -// tag::create-users[] -There are <> that you can use for specific -administrative purposes: `apm_system`, `beats_system`, `elastic`, `kibana_system`, -`logstash_system`, and `remote_monitoring_user`. - -// end::create-users[] - -Before you can use them, you must set their passwords: - -. Restart {es}. For example, if you installed {es} with a `.tar.gz` package, run -the following command from the {es} directory: -+ --- -["source","sh",subs="attributes,callouts"] ----------------------------------------------------------------------- -./bin/elasticsearch ----------------------------------------------------------------------- - -See {ref}/starting-elasticsearch.html[Starting {es}]. --- - -. Set the built-in users' passwords. -+ --- -// tag::create-users[] -Run the following command from the {es} directory: - -["source","sh",subs="attributes,callouts"] ----------------------------------------------------------------------- -./bin/elasticsearch-setup-passwords interactive ----------------------------------------------------------------------- -// end::create-users[] --- diff --git a/x-pack/docs/en/security/get-started-enable-security.asciidoc b/x-pack/docs/en/security/get-started-enable-security.asciidoc deleted file mode 100644 index bbe2999fc6753..0000000000000 --- a/x-pack/docs/en/security/get-started-enable-security.asciidoc +++ /dev/null @@ -1,35 +0,0 @@ -When you use the basic and trial licenses, the {es} {security-features} are -disabled by default. To enable them: - -. Stop {kib}. The method for starting and stopping {kib} varies depending on -how you installed it. For example, if you installed {kib} from an archive -distribution (`.tar.gz` or `.zip`), stop it by entering `Ctrl-C` on the command -line. See {kibana-ref}/start-stop.html[Starting and stopping {kib}]. - -. Stop {es}. For example, if you installed {es} from an archive distribution, -enter `Ctrl-C` on the command line. See -{ref}/stopping-elasticsearch.html[Stopping {es}]. - -. Add the `xpack.security.enabled` setting to the -`ES_PATH_CONF/elasticsearch.yml` file. -+ --- -TIP: The `ES_PATH_CONF` environment variable contains the path for the {es} -configuration files. If you installed {es} using archive distributions (`zip` or -`tar.gz`), it defaults to `ES_HOME/config`. If you used package distributions -(Debian or RPM), it defaults to `/etc/elasticsearch`. For more information, see -{ref}/settings.html[Configuring {es}]. - -For example, add the following setting: - -[source,yaml] ----- -xpack.security.enabled: true ----- - -TIP: If you have a basic or trial license, the default value for this setting is -`false`. If you have a gold or higher license, the default value is `true`. -Therefore, it is a good idea to explicitly add this setting to avoid confusion -about whether {security-features} are enabled. - --- diff --git a/x-pack/docs/en/security/get-started-kibana-users.asciidoc b/x-pack/docs/en/security/get-started-kibana-users.asciidoc deleted file mode 100644 index b701e77c7d6f4..0000000000000 --- a/x-pack/docs/en/security/get-started-kibana-users.asciidoc +++ /dev/null @@ -1,62 +0,0 @@ -When the {es} {security-features} are enabled, users must log in to {kib} -with a valid user ID and password. - -{kib} also performs some tasks under the covers that require use of the -built-in `kibana_system` user. - -. Configure {kib} to use the built-in `kibana_system` user and the password that you -created: - -** If you don't mind having passwords visible in your configuration file, -uncomment and update the following settings in the `kibana.yml` file in your -{kib} directory: -+ --- -TIP: If you installed {kib} using archive distributions (`zip` or -`tar.gz`), the `kibana.yml` configuration file is in `KIBANA_HOME/config`. If -you used package distributions (Debian or RPM), it's in `/etc/kibana`. For more -information, see {kibana-ref}/settings.html[Configuring {kib}]. - -For example, add the following settings: - -[source,yaml] ----- -elasticsearch.username: "kibana_system" -elasticsearch.password: "your_password" ----- - -Specify the password that you set with the `elasticsearch-setup-passwords` -command then save your changes to the file. --- - -** If you prefer not to put your user ID and password in the `kibana.yml` file, -store them in a keystore instead. Run the following commands to create the {kib} -keystore and add the secure settings: -+ --- -// tag::store-kibana-user[] -["source","sh",subs="attributes,callouts"] ----------------------------------------------------------------------- -./bin/kibana-keystore create -./bin/kibana-keystore add elasticsearch.username -./bin/kibana-keystore add elasticsearch.password ----------------------------------------------------------------------- - -When prompted, specify the `kibana_system` built-in user and its password for these -setting values. The settings are automatically applied when you start {kib}. -To learn more, see {kibana-ref}/secure-settings.html[Secure settings]. -// end::store-kibana-user[] --- - -. Restart {kib}. For example, if you installed -{kib} with a `.tar.gz` package, run the following command from the {kib} -directory: -+ --- -["source","sh",subs="attributes,callouts"] ----------------------------------------------------------------------- -./bin/kibana ----------------------------------------------------------------------- - -See {kibana-ref}/start-stop.html[Starting and stopping {kib}]. --- diff --git a/x-pack/docs/en/security/get-started-security.asciidoc b/x-pack/docs/en/security/get-started-security.asciidoc deleted file mode 100644 index ef9a291d3b49b..0000000000000 --- a/x-pack/docs/en/security/get-started-security.asciidoc +++ /dev/null @@ -1,377 +0,0 @@ -[role="xpack"] -[testenv="basic"] -[[security-getting-started]] -== Tutorial: Getting started with security - -In this tutorial, you learn how to secure a cluster by configuring users and -roles in {es}, {kib}, {ls}, and {metricbeat}. - -[discrete] -[[get-started-security-prerequisites]] -=== Before you begin - -. Install and configure {es}, {kib}, {ls}, and {metricbeat} as described in -{stack-gs}/get-started-elastic-stack.html[Getting started with the {stack}]. -+ --- -IMPORTANT: To complete this tutorial, you must install the default {es} and -{kib} packages, which include role-based access control (RBAC) and native -authentication {security-features}. When you install these products, they apply -basic licenses with no expiration dates. All of the subsequent steps in this -tutorial assume that you are using a basic license. For more information, see -{subscriptions} and {kibana-ref}/managing-licenses.html[License management]. - --- - -. Stop {ls}. The method for starting and stopping {ls} varies depending on whether -you are running it from the command line or running it as a service. For example, -if you are running {ls} from the command line, you can stop it by entering -`Ctrl-C`. See {logstash-ref}/shutdown.html[Shutting down {ls}]. - -. Stop {metricbeat}. For example, enter `Ctrl-C` on the command line where it is -running. - -. Launch the {kib} web interface by pointing your browser to port 5601. For -example, http://127.0.0.1:5601[http://127.0.0.1:5601]. - -[role="xpack"] -[[get-started-enable-security]] -=== Enable {es} {security-features} - -include::get-started-enable-security.asciidoc[] - -. Enable single-node discovery in the `ES_PATH_CONF/elasticsearch.yml` file. -+ --- -This tutorial involves a single node cluster, but if you had multiple -nodes, you would enable {es} {security-features} on every node in the cluster -and configure Transport Layer Security (TLS) for internode-communication, which -is beyond the scope of this tutorial. By enabling single-node discovery, we are -postponing the configuration of TLS. For example, add the following setting: - -[source,yaml] ----- -discovery.type: single-node ----- - -For more information, see -{ref}/bootstrap-checks.html#single-node-discovery[Single-node discovery]. --- - -When you enable {es} {security-features}, basic authentication is enabled by -default. To communicate with the cluster, you must specify a username and -password. Unless you <>, all requests -that don't include a user name and password are rejected. - -[role="xpack"] -[[get-started-built-in-users]] -=== Create passwords for built-in users - -include::get-started-builtin-users.asciidoc[] - -You need these built-in users in subsequent steps, so choose passwords that you -can remember! - -NOTE: This tutorial does not use the built-in `apm_system`, `logstash_system`, -`beats_system`, and `remote_monitoring_user` users, which are typically -associated with monitoring. For more information, see -{logstash-ref}/ls-security.html#ls-monitoring-user[Configuring credentials for {ls} monitoring] -and {metricbeat-ref}/monitoring.html[Monitoring {metricbeat}]. - -[role="xpack"] -[[get-started-kibana-user]] -=== Add the built-in user to {kib} - -include::get-started-kibana-users.asciidoc[] - -[role="xpack"] -[[get-started-authentication]] -=== Configure authentication - -Now that you've set up the built-in users, you need to decide how you want to -manage all the other users. - -The {stack} _authenticates_ users to ensure that they are valid. The -authentication process is handled by _realms_. You can use one or more built-in -realms, such as the native, file, LDAP, PKI, Active Directory, SAML, or Kerberos -realms. Alternatively, you can create your own custom realms. In this tutorial, -we'll use a native realm. - -In general, you configure realms by adding `xpack.security.authc.realms` -settings in the `elasticsearch.yml` file. However, the native realm is available -by default when no other realms are configured. Therefore, you don't need to do -any extra configuration steps in this tutorial. You can jump straight to -creating users! - -If you want to learn more about authentication and realms, see -<>. - -[role="xpack"] -[[get-started-users]] -=== Create users - -Let's create two users in the native realm. - -. Log in to {kib} with the `elastic` built-in user. - -. Open the menu. Go to *Stack Management > Security > Users*. -+ --- -[role="screenshot"] -image::security/images/management-builtin-users.jpg["User management screenshot in Kibana"] - -In this example, you can see a list of built-in users. --- - -. Click *Create user*. For example, create a user for yourself: -+ --- -[role="screenshot"] -image::security/images/create-user.jpg["Creating a user in Kibana"] - -You'll notice that when you create a user, you can assign it a role. Don't -choose a role yet--we'll come back to that in subsequent steps. --- - -. Click *Create user* and create a `logstash_internal` user. -+ --- -In {stack-gs}/get-started-elastic-stack.html[Getting started with the {stack}], -you configured {ls} to listen for {metricbeat} -input and to send the events to {es}. You therefore need to create a user -that {ls} can use to communicate with {es}. For example: - -[role="screenshot"] -image::security/images/create-logstash-user.jpg["Creating a {ls} user in {kib}"] --- - -[role="xpack"] -[[get-started-roles]] -=== Assign roles - -By default, all users can change their own passwords, get information about -themselves, and run the `authenticate` API. If you want them to do more than -that, you need to give them one or more _roles_. - -Each role defines a specific set of actions (such as read, create, or delete) -that can be performed on specific secured resources (such as data streams, indices, aliases, -documents, fields, or clusters). To help you get up and running, there are -built-in roles. - -Open the menu and go to *Stack Management > Security > Roles* to see them: - -[role="screenshot"] -image::security/images/management-roles.jpg["Role management screenshot in Kibana"] - -Select a role to see more information about its privileges. For example, select -the `kibana_system` role to see its list of cluster and index privileges. To -learn more, see <>. - -Let's assign the `kibana_admin` role to your user. Go back to the *Stack -Management > Security > Users* page and select your user. Add the `kibana_admin` -role and save the change. For example: - -[role="screenshot"] -image::security/images/assign-role.jpg["Assigning a role to a user in Kibana"] - -This user now has administrative access to all features in {kib}. -For more information about granting access to Kibana see -{kibana-ref}/xpack-security-authorization.html[Kibana authorization]. - -If you completed all of the steps in -{stack-gs}/get-started-elastic-stack.html[Getting started with the {stack}], you should -have {metricbeat} data stored in {es}. Let's create two roles that grant -different levels of access to that data. - -Go to the *Stack Management > Security > Roles* page and click *Create role*. - -Create a `metricbeat_reader` role that has `read` and `view_index_metadata` -privileges on the `metricbeat-*` indices: - -[role="screenshot"] -image::security/images/create-reader-role.jpg["Creating a role in Kibana"] - -Create a `metricbeat_writer` role that has `manage_index_templates` and `monitor` -cluster privileges, as well as `write`, `delete`, and `create_index` privileges -on the `metricbeat-*` indices: - -[role="screenshot"] -image::security/images/create-writer-role.jpg["Creating another role in Kibana"] - -Now go back to the *Stack -Management > Security > Users* page and assign these roles -to the appropriate users. Assign the `metricbeat_reader` role to your personal -user. Assign the `metricbeat_writer` role to the `logstash_internal` user. - -The list of users should now contain all of the built-in users as well as the -two you created. It should also show the appropriate roles for your users: - -[role="screenshot"] -image::security/images/management-users.jpg["User management screenshot in Kibana"] - -If you want to learn more about authorization and roles, see <>. - -[role="xpack"] -[[get-started-logstash-user]] -=== Add user information in {ls} - -In order for {ls} to send data successfully to {es}, you must configure its -authentication credentials in the {ls} configuration file. - -. Configure {ls} to use the `logstash_internal` user and the password that you -created: - -** If you don't mind having passwords visible in your configuration file, add -the following `user` and `password` settings in the `demo-metrics-pipeline.conf` -file in your {ls} directory: -+ --- -[source,ruby] ----- -... - -output { - elasticsearch { - hosts => "localhost:9200" - manage_template => false - index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}" - user => "logstash_internal" <1> - password => "your_password" <2> - } -} ----- -<1> Specify the `logstash_internal` user that you created earlier in this tutorial. -<2> Specify the password that you chose for this user ID. --- - -** If you prefer not to put your user ID and password in the configuration file, -store them in a keystore instead. -+ --- -Run the following commands to create the {ls} -keystore and add the secure settings: - -["source","sh",subs="attributes,callouts"] ----------------------------------------------------------------------- -set +o history -export LOGSTASH_KEYSTORE_PASS=mypassword <1> -set -o history -./bin/logstash-keystore create -./bin/logstash-keystore add ES_USER -./bin/logstash-keystore add ES_PWD ----------------------------------------------------------------------- -<1> You can optionally protect access to the {ls} keystore by storing a password -in an environment variable called `LOGSTASH_KEYSTORE_PASS`. For more information, -see {logstash-ref}/keystore.html#keystore-password[Keystore password]. - -When prompted, specify the `logstash_internal` user and its password for the -`ES_USER` and `ES_PWD` values. - -NOTE: The {ls} keystore differs from the {kib} keystore. Whereas the {kib} -keystore enables you to store `kibana.yml` settings by name, the {ls} keystore -enables you to create arbitrary names that you can reference in the {ls} -configuration. To learn more, see -{logstash-ref}/keystore.html[Secrets keystore for secure settings]. - -You can now use these `ES_USER` and `ES_PWD` keys in your configuration -file. For example, add the `user` and `password` settings in the -`demo-metrics-pipeline.conf` file as follows: - -[source,ruby] ----- -... - -output { - elasticsearch { - hosts => "localhost:9200" - manage_template => false - index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}" - user => "${ES_USER}" - password => "${ES_PWD}" - } -} ----- --- - -. Start {ls} by using the appropriate method for your environment. -+ --- -For example, to -run {ls} from a command line, go to the {ls} directory and enter the following -command: - -["source","sh",subs="attributes,callouts"] ----------------------------------------------------------------------- -./bin/logstash -f demo-metrics-pipeline.conf ----------------------------------------------------------------------- - -To start {ls} as a service, see -{logstash-ref}/running-logstash.html[Running {ls} as a service on Debian or RPM]. --- - -. If you were connecting directly from {metricbeat} to {es}, you would need -to configure authentication credentials for the {es} output in the {metricbeat} -configuration file. In -{stack-gs}/get-started-elastic-stack.html[Getting started with the {stack}], -however, you configured -{metricbeat} to send the data to {ls} for additional parsing, so no extra -settings are required in {metricbeat}. For more information, see -{metricbeat-ref}/securing-metricbeat.html[Securing {metricbeat}]. - -. Start {metricbeat} by using the appropriate method for your environment. -+ --- -For example, on macOS, run the following command from the {metricbeat} directory: - -["source","sh",subs="attributes,callouts"] ----------------------------------------------------------------------- -./metricbeat -e ----------------------------------------------------------------------- - -For more methods, see {metricbeat-ref}/metricbeat-starting.html[Starting {metricbeat}]. --- - -Wait a few minutes for new data to be sent from {metricbeat} to {ls} and {es}. - -[role="xpack"] -[[get-started-verify-users]] -=== View system metrics in {kib} - -Log in to {kib} with the user ID that has `metricbeat_reader` and `kibana_user` -roles (for example, `jdoe`). - -These roles enable the user to see the system metrics in {kib} (for example, on -the *Discover* page or in the -http://localhost:5601/app/kibana#/dashboard/Metricbeat-system-overview[{metricbeat} system overview dashboard]). - -[discrete] -[[gs-security-nextsteps]] -=== What's next? - -Congratulations! You've successfully set up authentication and authorization by -using the native realm. You learned how to create user IDs and roles that -prevent unauthorized access to the {stack}. - -Later, when you're ready to increase the number of nodes in your cluster, you'll -want to encrypt communications across the {stack}. To learn how, read -<>. - -For more detailed information about securing the {stack}, see: - -* <>. Encrypt -inter-node communications, set passwords for the built-in users, and manage your -users and roles. - -* {kibana-ref}/using-kibana-with-security.html[Configuring security in {kib}]. -Set the authentication credentials in {kib} and encrypt communications between -the browser and the {kib} server. - -* {logstash-ref}/ls-security.html[Configuring security in Logstash]. Set the -authentication credentials for Logstash and encrypt communications between -Logstash and {es}. - -* Configuring security in the Beats. Configure authentication -credentials and encrypt connections to {es}. For example, see -{filebeat-ref}/securing-filebeat.html[Configure {filebeat} to use {security-features}]. - -* {hadoop-ref}/security.html[Configuring {es} for Apache Hadoop to use secured transport]. diff --git a/x-pack/docs/en/security/securing-communications/node-certificates.asciidoc b/x-pack/docs/en/security/securing-communications/node-certificates.asciidoc deleted file mode 100644 index f5f2eed215578..0000000000000 --- a/x-pack/docs/en/security/securing-communications/node-certificates.asciidoc +++ /dev/null @@ -1,138 +0,0 @@ -[[node-certificates]] -==== Generating node certificates - -TLS requires X.509 certificates to perform encryption and authentication of the -application that is being communicated with. In order for the communication -between nodes to be truly secure, the certificates must be validated. The -recommended approach for validating certificate authenticity in an {es} cluster -is to trust the certificate authority (CA) that signed the certificate. By doing -this, as nodes are added to your cluster they just need to use a certificate -signed by the same CA and the node is automatically allowed to join the cluster. -Additionally, it is recommended that the certificates contain subject -alternative names (SAN) that correspond to the node's IP address and DNS name -so that hostname verification can be performed. - -The {ref}/certutil.html[`elasticsearch-certutil`] command simplifies the process -of generating certificates for the {stack}. It takes care of generating a CA and -signing certificates with the CA. It can be used interactively or in a silent -mode through the use of an input file. It also supports generation of -certificate signing requests (CSR), so that a commercial- or -organization-specific CA can be used to sign the certificates. - -NOTE: If you choose not to use `elasticsearch-certutil`, the certificates that -you obtain must allow for both `clientAuth` and `serverAuth` if the extended key -usage extension is present. The certificates need to be in PEM or PKCS#12 -format. Although not required, it is highly recommended that the certificate -contain the DNS names and/or IP addresses of the node so that hostname -verification can be used. - -. *Optional*: Create a certificate authority for your {es} cluster. -+ --- -Use the <> command: - -[source,shell] ----- -./bin/elasticsearch-certutil ca ----- - -You can configure the cluster to trust all nodes that have a certificate that -has been signed by this CA. - -The command outputs a single file with a default name of `elastic-stack-ca.p12`. -This file is a PKCS#12 keystore that contains the public certificate for your CA -and the private key that is used to sign the certificates for each node. - -The `elasticsearch-certutil` command also prompts you for a password to protect -the file and key. If you plan to add more nodes to your cluster in the future, -retain a copy of the file and remember its password. --- - -. Generate a certificate and private key for each node in your cluster. -+ --- -Use the <> command: - -[source,shell] ----- -./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 ----- -You are prompted for a password. You can enter a password for your -certificate and key, or you can leave the password blank. - -The output is a single PKCS#12 keystore that includes the node certificate, node -key, and CA certificate. - -IMPORTANT: Secure all output files, which contain the private keys -for your instance. - -The `elasticsearch-certutil` command generates certificates that have no -hostname information in them such as SAN fields. This -means that you can use the certificate for every node in your cluster, but you -must turn off hostname verification. - -If you want to use hostname verification within your cluster, run the -`elasticsearch-certutil cert` command once for each of your nodes and provide -the `--name`, `--dns` and `--ip` options. - -Alternatively, if you want to use a commercial or organization-specific CA, -you can use the <> command to -generate certificate signing requests (CSR) for the nodes in your cluster. --- - -. *Optional*: Generate additional certificates specifically for encrypting HTTP -client communications. -+ --- -Use the <> command: - -[source,shell] ----- -./bin/elasticsearch-certutil http ----- - -This command guides you through the process of generating the appropriate -certificates for use in {es} and {kib}. If you created a CA for your cluster, -you can re-use it by supplying its location when prompted. - -NOTE: The first question that the `elasticsearch-certutil` tool prompts you -with is whether you want to generate a Certificate Signing Request (CSR). -Answer `y` if a trusted authority, such as in internal security team or a -commercial certificate authority, will sign your certificates. Answer `n` if -you have access to a trusted CA and will sign your own certificates. - -The output is a `.zip` file containing one directory each for both Elasticsearch -and Kibana: - -[source,shell] ----- -/elasticsearch -|_ README.txt -|_ http.p12 -|_ sample-elasticsearch.yml - -/kibana -|_ README.txt -|_ elasticsearch-ca.pem -|_ sample-kibana.yml ----- --- - -. Copy the node certificate to the appropriate locations. - -.. Create a folder in the configuration directory on each {es} node to contain -your security certificates. For example, create a `certs` folder in the -`/home/es/config/certs` directory. -+ -NOTE: The <> varies -depending on your {es} installation. - -.. On each node, copy the certificate that you created into the `/certs` -directory. This certificate is typically a single `.p12` (PKCS#12) file, such -as `elastic-certificates.p12`. - -.. If you generated HTTP certificates, copy the `http.p12` file into the -`certs` directory as well. - -.. For each additional Elastic product that you want to configure, copy the -certificates to the relevant configuration directory. diff --git a/x-pack/docs/en/security/securing-communications/securing-elasticsearch.asciidoc b/x-pack/docs/en/security/securing-communications/securing-elasticsearch.asciidoc deleted file mode 100644 index 9d21b0bcfca68..0000000000000 --- a/x-pack/docs/en/security/securing-communications/securing-elasticsearch.asciidoc +++ /dev/null @@ -1,42 +0,0 @@ -[role="xpack"] -[[configuring-tls]] -=== Encrypting communications in {es} - -{stack} {security-features} enable you to encrypt traffic to, from, and within -your {es} cluster. Connections are secured using Transport Layer Security -(TLS/SSL). - -WARNING: Clusters that do not have encryption enabled send all data in plain text -including passwords. If the {es} {security-features} are enabled, unless you -have a trial license, you must configure SSL/TLS for internode-communication. - -To enable encryption, you need to perform the following steps on each node in -the cluster: - -. Verify that the `xpack.security.enabled` setting is `true`. For more -information, see <>. - -. <>. - -. Configure each node to: -.. Required: <>. -.. Recommended: <>. - -. If you are using Active Directory user authentication, -<>. - -. If you are using LDAP user authentication, -<>. - -For more information about encrypting communications across the Elastic Stack, -see <>. - -include::node-certificates.asciidoc[] - -include::tls-transport.asciidoc[] - -include::tls-http.asciidoc[] - -include::tls-ad.asciidoc[] - -include::tls-ldap.asciidoc[] diff --git a/x-pack/docs/en/security/securing-communications/tls-transport.asciidoc b/x-pack/docs/en/security/securing-communications/tls-transport.asciidoc deleted file mode 100644 index 0ae1981ee1049..0000000000000 --- a/x-pack/docs/en/security/securing-communications/tls-transport.asciidoc +++ /dev/null @@ -1,109 +0,0 @@ -[role="xpack"] -[[tls-transport]] -==== Encrypting communications between nodes in a cluster - -The transport networking layer is used for internal communication between nodes -in a cluster. When {security-features} are enabled, you must use TLS to ensure -that communication between the nodes is encrypted. - -. <>. - -. Enable TLS and specify the information required to access the node’s -certificate. - -** If the signed certificate is in PKCS#12 format, add the following information to the -`elasticsearch.yml` file on each node: -+ --- -[source,yaml] ------------------------------------------------------------ -xpack.security.transport.ssl.enabled: true -xpack.security.transport.ssl.verification_mode: certificate <1> -xpack.security.transport.ssl.keystore.path: elastic-certificates.p12 <2> -xpack.security.transport.ssl.truststore.path: elastic-certificates.p12 <3> ------------------------------------------------------------ -<1> If you used the `--dns` or `--ip` options with the `elasticsearch-certutil cert` command -and you want to enable strict hostname checking, set the verification mode to -`full`. For a description of these values, see <>. - -<2> If you created a separate certificate for each node, then you might need to -customize this path on each node. If the filename matches the node name, you can -use the `${node.name}.p12` format, for example. -<3> The `elasticsearch-certutil` outputs a PKCS#12 keystore which includes the -CA certificate as a trusted certificate entry. This allows for the keystore to -also be used as a truststore. In this case, the path value should match -the `keystore.path` value. -Note, however, that this is not the general rule. There are keystores that cannot be -used as truststores, only -{ref}/security-settings.html#pkcs12-truststore-note[specifically crafted ones can] --- - -** If the certificate is in PEM format, add the following information to the -`elasticsearch.yml` file on each node: -+ --- -[source, yaml] --------------------------------------------------- -xpack.security.transport.ssl.enabled: true -xpack.security.transport.ssl.verification_mode: certificate <1> -xpack.security.transport.ssl.key: /home/es/config/node01.key <2> -xpack.security.transport.ssl.certificate: /home/es/config/node01.crt <3> -xpack.security.transport.ssl.certificate_authorities: [ "/home/es/config/ca.crt" ] <4> --------------------------------------------------- -<1> If you used the `--dns` or `--ip` options with the `elasticsearch-certutil cert` command -and you want to enable strict hostname checking, set the verification mode to -`full`. For a description of these values, see <>. -<2> The full path to the node key file. This must be a location within the - {es} configuration directory. -<3> The full path to the node certificate. This must be a location within the - {es} configuration directory. -<4> An array of paths to the CA certificates that should be trusted. These paths - must be a location within the {es} configuration directory. --- - -. If you secured the node's certificate with a password, add the password to -your {es} keystore: - -** If the signed certificate is in PKCS#12 format, use the following commands: -+ --- -[source,shell] ------------------------------------------------------------ -bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password - -bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password ------------------------------------------------------------ --- - -** If the certificate is in PEM format, use the following commands: -+ --- -[source,shell] ------------------------------------------------------------ -bin/elasticsearch-keystore add xpack.security.transport.ssl.secure_key_passphrase ------------------------------------------------------------ --- - -. Restart {es}. -+ --- -You must perform a full cluster restart. Nodes which are configured to use TLS -cannot communicate with nodes that are using unencrypted networking (and -vice-versa). After enabling TLS you must restart all nodes in order to maintain -communication across the cluster. --- - -[NOTE] -=============================== -* All TLS-related node settings are considered to be highly sensitive and -therefore are not exposed via the -{ref}/cluster-nodes-info.html#cluster-nodes-info[nodes info API] For more -information about any of these settings, see <>. - -* {es} monitors all files such as certificates, keys, keystores, or truststores -that are configured as values of TLS-related node settings. If you update any of -these files (for example, when your hostnames change or your certificates are -due to expire), {es} reloads them. The files are polled for changes at -a frequency determined by the global {es} `resource.reload.interval.high` -setting, which defaults to 5 seconds. -=============================== diff --git a/x-pack/docs/en/security/securing-communications/tutorial-tls-addnodes.asciidoc b/x-pack/docs/en/security/securing-communications/tutorial-tls-addnodes.asciidoc deleted file mode 100644 index 6604a86e96261..0000000000000 --- a/x-pack/docs/en/security/securing-communications/tutorial-tls-addnodes.asciidoc +++ /dev/null @@ -1,188 +0,0 @@ -[role="xpack"] -[testenv="basic"] -[[encrypting-communications-hosts]] -=== Add nodes to your cluster - -You can add more nodes to your cluster and optionally designate specific -purposes for each node. For example, you can allocate master nodes, data nodes, -ingest nodes, machine learning nodes, and dedicated coordinating nodes. For -details about each node type, see {ref}/modules-node.html[Nodes]. - -Let's add two nodes to our cluster! - -. Install two additional copies of {es}. It's possible to run multiple {es} -nodes using a shared installation. In this tutorial, however, we're keeping -things simple by using the `zip` or `tar.gz` packages and by putting each copy -in a separate folder. You can simply repeat the steps that you used to install -{es} in the -{stack-gs}/get-started-elastic-stack.html#install-elasticsearch[Getting started with the {stack}] -tutorial. - -. Generate certificates for the two new nodes. -+ --- -For example, run the following command: - -["source","sh",subs="attributes,callouts"] ----------------------------------------------------------------------- -./bin/elasticsearch-certutil cert \ ---ca elastic-stack-ca.p12 \ <1> ---multiple ----------------------------------------------------------------------- -<1> Use the certificate authority that you created in <>. - -You are prompted for information about each new node. Specify `node-2` and -`node-3` for the instance names. For the purposes of this tutorial, specify the -same IP address (`127.0.0.1,::1`) and DNS name (`localhost`) for each node. - -You are prompted to enter the password for your CA. You are also prompted to -create a password for each certificate. - -By default, the command produces a zip file named `certificate-bundle.zip`, -which contains the generated certificates and keys. --- - -. Decompress the `certificate-bundle.zip` file. For example: -+ --- -["source","sh",subs="attributes,callouts"] ----------------------------------------------------------------------- -unzip certificate-bundle.zip - -Archive: certificate-bundle.zip - creating: node-2/ - inflating: node-2/node-2.p12 - creating: node-3/ - inflating: node-3/node-3.p12 ----------------------------------------------------------------------- - -The `certificate-bundle.zip` file contains a folder for each of your nodes. Each -folder contains a single PKCS#12 keystore that includes a node certificate, -node key, and CA certificate. --- - -. Create a folder to contain certificates in the configuration directory of each -{es} node. For example, create a `certs` folder in the `config` directory. - -. Copy the appropriate certificate to the configuration directory on each node. -For example, copy the `node-2.p12` file into the `config/certs` directory on the -second node and the `node-3.p12` into the `config/certs` directory on the third -node. - -. Specify the name of the cluster and give each node a unique name. -+ --- -For example, add the following settings to the `ES_PATH_CONF/elasticsearch.yml` -file on the second node: - -[source,yaml] ----- -cluster.name: test-cluster -node.name: node-2 ----- - -Add the following settings to the `ES_PATH_CONF/elasticsearch.yml` -file on the third node: - -[source,yaml] ----- -cluster.name: test-cluster -node.name: node-3 ----- - -NOTE: In order to join the same cluster as the first node, they must share the -same `cluster.name` value. - --- - -. (Optional) Provide seed addresses to help your nodes discover other nodes with -which to form a cluster. -+ --- -For example, add the following setting in the `ES_PATH_CONF/elasticsearch.yml` -file: - -[source,yaml] ----- -discovery.seed_hosts: ["localhost"] ----- - -The default value for this setting is `127.0.0.1, [::1]`, therefore it isn't -actually required in this tutorial. When you want to form a cluster with nodes -on other hosts, however, you must use this setting to provide a list of -master-eligible nodes to seed the discovery process. For more information, see -{ref}/modules-discovery-hosts-providers.html[Discovery]. --- - -. On each node, enable TLS for transport communications. You must also configure -each node to identify itself using its signed certificate. -+ --- -include::tutorial-tls-internode.asciidoc[tag=enable-tls] --- - -. On each node, store the password for the PKCS#12 file in the {es} keystore. -+ --- -include::tutorial-tls-internode.asciidoc[tag=secure-passwords] - -On the second node, supply the password that you created for the `node-2.p12` -file. On the third node, supply the password that you created for the -`node-3.p12` file. --- - -. Start each {es} node. For example, if you installed {es} with a `.tar.gz` -package, run the following command from each {es} directory: -+ --- -["source","sh",subs="attributes,callouts"] ----------------------------------------------------------------------- -./bin/elasticsearch ----------------------------------------------------------------------- - -See {ref}/starting-elasticsearch.html[Starting {es}]. - -If you encounter errors, you can see some common problems and solutions in -<>. --- - -. Verify that your cluster now contains three nodes. -+ --- -For example, log in to {kib} with the `elastic` built-in user. Go to -*Dev Tools > Console* and run the {ref}/cluster-health.html[cluster health API]: - -[source,console] ----------------------------------- -GET _cluster/health ----------------------------------- - -Confirm the `number_of_nodes` in the response from this API. - -You can also use the {ref}/cat-nodes.html[cat nodes API] to identify the master -node: - -[source,console] ----------------------------------- -GET _cat/nodes?v=true ----------------------------------- - -The node that has an asterisk(*) in the `master` column is the elected master -node. --- - -Now that you have multiple nodes, your data can be distributed across the -cluster in multiple primary and replica shards. For more information about the -concepts of clusters, nodes, and shards, see -{ref}/getting-started.html[Getting started with {es}]. - -[discrete] -[[encrypting-internode-nextsteps]] -=== What's next? - -Congratulations! You've encrypted communications between the nodes in your -cluster and can pass the -<>. - -If you want to encrypt communications between other products in the {stack}, see -<>. diff --git a/x-pack/docs/en/security/securing-communications/tutorial-tls-certificates.asciidoc b/x-pack/docs/en/security/securing-communications/tutorial-tls-certificates.asciidoc deleted file mode 100644 index c806e352b1ca1..0000000000000 --- a/x-pack/docs/en/security/securing-communications/tutorial-tls-certificates.asciidoc +++ /dev/null @@ -1,77 +0,0 @@ -[role="xpack"] -[testenv="basic"] -[[encrypting-communications-certificates]] -=== Generate certificates - -In a secured cluster, {es} nodes use certificates to identify themselves when -communicating with other nodes. - -The cluster must validate the authenticity of these certificates. The -recommended approach is to trust a specific certificate authority (CA). Thus -when nodes are added to your cluster they just need to use a certificate signed -by the same CA. - -. Generate a certificate authority for your cluster. -+ --- -Run the following command: - -["source","sh",subs="attributes,callouts"] ----------------------------------------------------------------------- -./bin/elasticsearch-certutil ca ----------------------------------------------------------------------- - -You are prompted for an output filename and a password. In this tutorial, we'll -use the default filename (`elastic-stack-ca.p12`). - -The output file is a PKCS#12 keystore that contains the public certificate for -your certificate authority and the private key that is used to sign the node -certificates. - -TIP: We'll need to use this file again when we add nodes to the cluster, so -remember its location and password. Ideally you should store the file securely, -since it holds the key to your cluster. - -For more information about this command, see -{ref}/certutil.html[elasticsearch-certutil]. --- - -. Create a folder to contain certificates in the configuration directory of your -{es} node. For example, create a `certs` folder in the `config` directory. - -. Generate certificates and private keys for the first node in your cluster. -+ --- -Run the following command: - -["source","sh",subs="attributes,callouts"] ----------------------------------------------------------------------- -./bin/elasticsearch-certutil cert \ ---ca elastic-stack-ca.p12 \ <1> ---dns localhost \ <2> ---ip 127.0.0.1,::1 <3> ---out config/certs/node-1.p12 <4> ----------------------------------------------------------------------- -<1> The `--ca` parameter contains the name of certificate authority that you -generated for this cluster. -<2> The `--dns` parameter contains a comma-separated list of DNS names for the -node. -<3> The `--ip` parameter contains a comma-separated list of IP addresses for the -node. -<4> The `--out` parameter contains the name and location of the generated -certificate. Ideally the file name matches the `node.name` value in the -`elasticsearch.yml` file. - -You are prompted to enter the password for your CA. You are also prompted to -create a password for the certificate. - -The output file is a PKCS#12 keystore that includes a node certificate, node key, -and CA certificate. --- - -TIP: The {ref}/certutil.html[elasticsearch-certutil] command has a lot more -options. For example, it can generate Privacy Enhanced Mail (PEM) formatted -certificates and keys. It can also generate certificate signing requests (CSRs) -that you can use to obtain signed certificates from a commercial or -organization-specific certificate authority. However, those options are not -covered in this tutorial. diff --git a/x-pack/docs/en/security/securing-communications/tutorial-tls-internode.asciidoc b/x-pack/docs/en/security/securing-communications/tutorial-tls-internode.asciidoc deleted file mode 100644 index a45e06c5b3bc5..0000000000000 --- a/x-pack/docs/en/security/securing-communications/tutorial-tls-internode.asciidoc +++ /dev/null @@ -1,177 +0,0 @@ -[role="xpack"] -[testenv="trial"] -[[encrypting-internode]] -=== Encrypt internode communications - -Now that we've generated a certificate authority and certificates, let's update -the cluster to use these files. - -IMPORTANT: When you enable {es} {security-features}, unless you have a trial -license, you must use Transport Layer Security (TLS) to encrypt internode -communication. By following the steps in this tutorial, you learn how -to meet the minimum requirements to pass the -<>. - -. (Optional) Name the cluster. -+ --- -For example, add the <> setting in the -`ES_PATH_CONF/elasticsearch.yml` file: - -[source,yaml] ----- -cluster.name: test-cluster ----- - -TIP: The `ES_PATH_CONF` environment variable contains the path for the {es} -configuration files. If you installed {es} using archive distributions (`zip` or -`tar.gz`), it defaults to `ES_HOME/config`. If you used package distributions -(Debian or RPM), it defaults to `/etc/elasticsearch`. For more information, see -<>. - -The default cluster name is `elasticsearch`. You should choose a unique name, -however, to ensure that your nodes join the right cluster. --- - -. (Optional) Name the {es} node. -+ --- -For example, add the <> setting in the -`ES_PATH_CONF/elasticsearch.yml` file: - -[source,yaml] ----- -node.name: node-1 ----- - -In this tutorial, the cluster will consist of three nodes that exist on the same -machine and share the same (loopback) IP address and hostname. Therefore, we -must give each node a unique name. - -This step is also necessary if you want to use the `node.name` value to define -the location of certificates in subsequent steps. --- - -. Disable single-node discovery. -+ --- -To enable {es} to form a multi-node cluster, use the default value for the -`discovery.type` setting. If that setting exists in your -`ES_PATH_CONF/elasticsearch.yml` file, remove it. --- - -. (Optional) If you are starting the cluster for the first time, specify the -initial set of master-eligible nodes. -+ --- -For example, add the following setting in the `ES_PATH_CONF/elasticsearch.yml` -file: - -[source,yaml] ----- -cluster.initial_master_nodes: ["node-1"] ----- - -If you start an {es} node without configuring this setting or any other -discovery settings, it will start up in development mode and auto-bootstrap -itself into a new cluster. - -TIP: If you are starting a cluster with multiple master-eligible nodes for the -first time, add all of those node names to the `cluster.initial_master_nodes` -setting. - -See <> and -<>. --- - -. Enable Transport Layer Security (TLS/SSL) for transport (internode) -communications. -+ --- -// tag::enable-tls[] -For example, add the following settings in the `ES_PATH_CONF/elasticsearch.yml` -file: - -[source,yaml] ----- -xpack.security.enabled: true -xpack.security.transport.ssl.enabled: true -xpack.security.transport.ssl.keystore.path: certs/${node.name}.p12 <1> -xpack.security.transport.ssl.truststore.path: certs/${node.name}.p12 ----- -<1> If the file name for your certificate does not match the `node.name` value, -you must put the appropriate file name in the `elasticsearch.yml` file. -// end::enable-tls[] - -NOTE: The PKCS#12 keystore that is output by the `elasticsearch-certutil` can be -used as both a keystore and a truststore. If you use other tools to manage and -generate your certificates, you might have different values for these settings, -but that scenario is not covered in this tutorial. - -For more information, see <> and -<>. --- - -. Store the password for the PKCS#12 file in the {es} keystore. -+ --- -// tag::secure-passwords[] -For example, run the following commands: - -["source","sh",subs="attributes,callouts"] ----------------------------------------------------------------------- -./bin/elasticsearch-keystore create <1> -./bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password -./bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password ----------------------------------------------------------------------- -<1> If the {es} keystore already exists, this command asks whether you want to -overwrite it. You do not need to overwrite it; you can simply add settings to -your existing {es} keystore. -// end::secure-passwords[] - -You are prompted to supply the password that you created for the `node-1.p12` -file. We are using this file for both the transport TLS keystore and truststore, -therefore supply the same password for both of these settings. --- - -. <>. -+ --- -For example, if you installed {es} with a `.tar.gz` package, run the following -command from the {es} directory: - -["source","sh",subs="attributes,callouts"] ----------------------------------------------------------------------- -./bin/elasticsearch ----------------------------------------------------------------------- --- - -. Create passwords for the built-in users and configure {kib} to use them. -+ --- -NOTE: If you already configured passwords for these users in other tutorials, -you can skip this step. - -include::{xes-repo-dir}/security/get-started-builtin-users.asciidoc[tag=create-users] - -After you setup the password for the `kibana_system` built-in user, -<>. - -For example, run the following commands to create the {kib} keystore and add the -`kibana_system` built-in user and its password in secure settings: - -include::{xes-repo-dir}/security/get-started-kibana-users.asciidoc[tag=store-kibana-user] --- - -. Start {kib}. -+ --- -For example, if you installed {kib} with a `.tar.gz` package, run the following -command from the {kib} directory: -["source","sh",subs="attributes,callouts"] ----------------------------------------------------------------------- -./bin/kibana ----------------------------------------------------------------------- - -See {kibana-ref}/start-stop.html[Starting and stopping {kib}]. --- diff --git a/x-pack/docs/en/security/securing-communications/tutorial-tls-intro.asciidoc b/x-pack/docs/en/security/securing-communications/tutorial-tls-intro.asciidoc deleted file mode 100644 index d2c4b292483cc..0000000000000 --- a/x-pack/docs/en/security/securing-communications/tutorial-tls-intro.asciidoc +++ /dev/null @@ -1,47 +0,0 @@ -[role="xpack"] -[testenv="basic"] -[[encrypting-internode-communications]] -== Tutorial: Encrypting communications - -In the {stack-gs}/get-started-elastic-stack.html[Getting started with the {stack}] -and <> tutorials, we -used a cluster with a single {es} node to get up and running with the {stack}. - -You can add as many nodes as you want in a cluster but they must be able to -communicate with each other. The communication between nodes in a cluster is -handled by the <>. To secure your cluster, -you must ensure that the internode communications are encrypted. - -NOTE: In this tutorial, we add more nodes by installing more copies of {es} on -the same machine. By default, {es} binds to loopback addresses for HTTP and -transport communication. That is fine for the purposes of this tutorial and for -downloading and experimenting with {es} in a test or development environment. -When you are deploying a production environment, however, you are generally -adding nodes on different machines so that your cluster is resilient to outages -and avoids data loss. In a production scenario, there are additional -requirements that are not covered in this tutorial. See -{ref}/bootstrap-checks.html#dev-vs-prod-mode[Development vs production mode] and -{ref}/add-elasticsearch-nodes.html[Adding nodes to your cluster]. - -[discrete] -[[encrypting-internode-prerequisites]] -=== Before you begin - -Ideally, you should do this tutorial after you complete the -{stack-gs}/get-started-elastic-stack.html[Getting started with the {stack}] and -<> tutorials. - -At a minimum, you must install and configure {es} and {kib} in a cluster with a -single {es} node. In particular, this tutorial provides instructions for adding -nodes that work with the `zip` and `tar.gz` packages. - -IMPORTANT: To complete this tutorial, you must install the default {es} and -{kib} packages, which include the encrypted communications {security-features}. -When you install these products, they apply basic licenses with no expiration -dates. All of the subsequent steps in this tutorial assume that you are using a -basic license. For more information, see {subscriptions} and -{kibana-ref}/managing-licenses.html[License management]. - -include::tutorial-tls-certificates.asciidoc[] -include::tutorial-tls-internode.asciidoc[] -include::tutorial-tls-addnodes.asciidoc[]