Skip to content

Latest commit

 

History

History
466 lines (367 loc) · 14.7 KB

get-started-docker.asciidoc

File metadata and controls

466 lines (367 loc) · 14.7 KB
Raw

Running the {stack} ("ELK") on Docker

Running the {stack} on Docker

The Elastic Docker registry contains Docker images for all the products in the {stack}: https://www.docker.elastic.co/.

You can start the {stack} in Docker with security enabled and configured by default. This option is great for quickly getting started with {es} and {kib} for testing and development.

You can also start the {stack} with Docker Compose to create a secured, multi-node cluster with a connected {kib} instance. This option results in a more resilient cluster with greater capacity and reliability.

Start {es} and {kib} in Docker

Starting in {es} 8.0, security is enabled and configured by default.

If you’re starting a single-node {es} cluster in a Docker container, security will be automatically enabled and configured for you. When you start {es} for the first time, the following security configuration occurs automatically:

  • {ref}/configuring-stack-security.html#stack-security-certificates[Certificates and keys] are generated for the transport and HTTP layers.

  • The Transport Layer Security (TLS) configuration settings are written to elasticsearch.yml.

  • A password is generated for the elastic user.

  • An enrollment token is generated for {kib}.

  • An enrollment token is generated for other {es} nodes.

You can then {kibana-ref}/docker.html[start {kib}] and enter the enrollment token, which is valid for 30 minutes. This token automatically applies the security settings from your {es} cluster, authenticates to {es} with the kibana service account, and writes the security configuration to kibana.yml.

Prerequisites

  1. Install the appropriate Docker application for your operating system.

  2. Pull the {es} Docker image.

    docker pull docker.elastic.co/elasticsearch/elasticsearch:{version}

  3. Pull the {kib} Docker image.

    docker pull docker.elastic.co/kibana/kibana:{version}

Start a single-node {es} cluster and enroll {kib}

The following command starts a single-node {es} cluster for development or testing.

  1. Create an elastic network for your containers.

    docker network create elastic

  2. Start {es} in Docker. A password is generated for the elastic user and output to the terminal, plus enrollment tokens for enrolling {kib} and adding additional nodes to your cluster.

    docker run --name es01 --net elastic -p 9200:9200 -it docker.elastic.co/elasticsearch/elasticsearch:{version}
    Tip
    		 You might need to scroll back a bit in the terminal to view the password and enrollment token.

  3. Copy the generated password and enrollment token and save them in a secure location. These values are shown only when you start {es} for the first time.

  4. Copy the http_ca.crt security certificate from your Docker container to your local machine.

    docker cp es01:/usr/share/elasticsearch/config/certs/http_ca.crt .

  5. Open a new terminal and verify that you can connect to your {es} cluster by making an authenticated call, using the http_ca.crt file that you copied from your Docker container. Enter the password for the elastic user when prompted.

    curl --cacert http_ca.crt -u elastic https://localhost:9200

    If the connection is successful, the command returns a response like this:

    {
  "name" : "Cp8oag6",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "AT69_T_DTp-1qgIJlatQqA",
  "version" : {
    "number" : "{elasticsearch_version}",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "f27399d",
    "build_date" : "2021-11-04T12:35:26.989068569Z",
    "build_snapshot" : false,
    "lucene_version" : "9.0.0",
    "minimum_wire_compatibility_version" : "7.16.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

  6. In a new terminal session, start {kib} and connect it to your {es} container:

    docker run --name kibana --net elastic -p 5601:5601 docker.elastic.co/kibana/kibana:{version}

    When you start {kib}, a unique link is output to your terminal.

  7. To access {kib}, click the generated link in your terminal.

Note

If the generated link is to host 0.0.0.0 you may have to replace it with 127.0.0.1

  1. In your browser, paste the enrollment token that you copied when starting {es} and click the button to connect your {kib} instance with {es}.

  2. Log in to {kib} as the elastic user with the password that was generated when you started {es}.

Enroll additional nodes

When you start {es} for the first time, the installation process configures a single-node cluster by default. This process also generates an enrollment token and prints it to your terminal. If you want a node to join an existing cluster, start the new node with the generated enrollment token.

Generating enrollment tokens

The enrollment token is valid for 30 minutes. If you need to generate a new enrollment token, run the {ref}/create-enrollment-token.html[elasticsearch-create-enrollment-token] tool on your existing node. This tool is available in the {es} bin directory of the Docker container.

For example, run the following command on the existing es01 node to generate an enrollment token for new {es} nodes:

docker exec -it es01 /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node

  1. In the terminal where you started your first node, copy the generated enrollment token for adding new {es} nodes.

  2. On your new node, start {es} and include the generated enrollment token.

    docker run -e ENROLLMENT_TOKEN="" --name es02 --net elastic -it docker.elastic.co/elasticsearch/elasticsearch:{version}

    {es} is now configured to join the existing cluster.

If you experience issues where the container where your first node is running exits when your second node starts, explicitly set values for the JVM heap size. To {ref}/docker.html#docker-set-heap-size[manually configure the heap size], include the ES_JAVA_OPTS variable and set values for -Xms and -Xmx when starting each node. For example, the following command starts node es02 and sets the minimum and maximum JVM heap size to 1 GB:

docker run -e ES_JAVA_OPTS="-Xms1g -Xmx1g" -e ENROLLMENT_TOKEN="" --name es02 -p 9201:9200 --net elastic -it docker.elastic.co/elasticsearch/elasticsearch:{version}

Generate passwords and enrollment tokens

If you need to reset the password for the elastic user or other built-in users, run the {ref}/reset-password.html[elasticsearch-reset-password] tool. This tool is available in the {es} bin directory of the Docker container.

For example, to reset the password for the elastic user:

docker exec -it es01 /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic

If you need to generate new enrollment tokens for {kib} or {es} nodes, run the {ref}/create-enrollment-token.html[elasticsearch-create-enrollment-token] tool. This tool is available in the {es} bin directory of the Docker container.

For example, to generate a new enrollment token for {kib}:

docker exec -it es01 /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana

Remove Docker containers

To remove the containers and their network, run:

docker network rm elastic
docker rm es01
docker rm kibana

Start the {stack} with Docker Compose

To get a multi-node {es} cluster and {kib} up and running in Docker with security enabled, you can use Docker Compose.

This configuration provides a simple method of starting a secured cluster that you can use for development before building a distributed deployment with multiple hosts.

Prerequisites

Install the appropriate Docker application for your operating system.

If you’re running on Linux, install Docker Compose.

Note

Make sure that Docker is allotted at least 4GB of memory. In Docker Desktop, you configure resource usage on the Advanced tab in Preferences (macOS) or Settings (Windows).

Prepare the environment

Create the following configuration files in a new, empty directory. These files are also available from the elastic/elasticsearch repository on GitHub.

.env

The .env file sets environment variables that are used when you run the docker-compose.yml configuration file. Ensure that you specify a strong password for the elastic and kibana_system users with the ELASTIC_PASSWORD and KIBANA_PASSWORD variables. These variable are referenced by the docker-compose.yml file.

link:{es-repo-dir}/setup/install/docker/.env[role=include]
docker-compose.yml

This docker-compose.yml file creates a three-node secure {es} cluster with authentication and network encryption enabled, and a {kib} instance securely connected to it.

Exposing ports

This configuration exposes port 9200 on all network interfaces. Because of how Docker handles ports, a port that isn’t bound to localhost leaves your {es} cluster publicly accessible, potentially ignoring any firewall settings. If you don’t want to expose port 9200 to external hosts, set the value for ES_PORT in the .env file to something like 127.0.0.1:9200. {es} will then only be accessible from the host machine itself.

link:{es-repo-dir}/setup/install/docker/docker-compose.yml[role=include]

Start your cluster with security enabled and configured

  1. Modify the .env file and enter strong password values for both the ELASTIC_PASSWORD and KIBANA_PASSWORD variables.

    Note
    		 You must use the ELASTIC_PASSWORD value for further interactions with the cluster. The KIBANA_PASSWORD value is only used internally when configuring {kib}.

  2. Create and start the three-node {es} cluster and {kib} instance:

    docker-compose up -d

  3. When the deployment has started, open a browser and navigate to http://localhost:5601 to access {kib}, where you can load sample data and interact with your cluster. Log in as the elastic user with the ELASTIC_PASSWORD value to get access.

Stop and remove the deployment

When you’re done experimenting, you can remove the network, containers, and volumes:

docker-compose down -v

Load settings from a file

Specifying settings for {es} and {kib} directly in the Docker Compose file is a convenient way to get started, but loading settings from a file is preferable after you get past the experimental stage.

For example, to use a custom es01.yml as the configuration file for the es01 {es} node, you can create a bind mount in the volumes section for the es01 service.

    volumes:
      - ./es01.yml:/usr/share/elasticsearch/config/elasticsearch.yml
      - ...

Similarly, to load {kib} settings from a file, you can add the following mount in the volumes section for the kibana service.

    volumes:
      - ./kibana.yml:/usr/share/kibana/config/kibana.yml
      - ...

Product-specific instructions for Docker

See the product-specific documentation for information about running a specific Elastic product in Docker:

  • {ref}/docker.html[Install {es} with Docker]

  • {fleet-guide}/elastic-agent-container.html[Run Elastic Agent in a container]