Skip to content

Commit

Permalink
Upgrade Elasticsearch to version 5.2.2
Browse files Browse the repository at this point in the history
  • Loading branch information
@pires
pires committed Mar 1, 2017
1 parent 02ae85b commit c33b218
Show file tree
Hide file tree
Showing 5 changed files with 63 additions and 63 deletions.
118 changes: 59 additions & 59 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# kubernetes-elasticsearch-cluster
Elasticsearch (5.2.0) cluster on top of Kubernetes made easy.
Elasticsearch (5.2.2) cluster on top of Kubernetes made easy.

Elasticsearch best-practices recommend to separate nodes in three roles:
* `Master` nodes - intended for clustering management only, no data, no HTTP API
Expand Down Expand Up @@ -46,58 +46,58 @@ Now, I leave up to you how to validate the cluster, but a first step is to wait

```
$ kubectl get svc,deployment,pods
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
svc/elasticsearch 10.100.203.98 <pending> 9200:32057/TCP 4m
svc/elasticsearch-discovery 10.100.131.27 <none> 9300/TCP 4m
svc/kubernetes 10.100.0.1 <none> 443/TCP 12m
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
svc/elasticsearch 10.100.18.90 <pending> 9200:30565/TCP 32m
svc/elasticsearch-discovery 10.100.132.130 <none> 9300/TCP 32m
svc/kubernetes 10.100.0.1 <none> 443/TCP 43m
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
deploy/es-client 2 2 2 2 2m
deploy/es-data 2 2 2 2 2m
deploy/es-client 2 2 2 2 1m
deploy/es-data 2 2 2 2 1m
deploy/es-master 3 3 3 3 4m
NAME READY STATUS RESTARTS AGE
po/es-client-583869315-11mjj 1/1 Running 0 2m
po/es-client-583869315-q9wsj 1/1 Running 0 2m
po/es-data-903871959-ctbsj 1/1 Running 0 2m
po/es-data-903871959-mw0tj 1/1 Running 0 2m
po/es-master-2173384742-40g9j 1/1 Running 0 4m
po/es-master-2173384742-d2m7j 1/1 Running 0 4m
po/es-master-2173384742-q7ckf 1/1 Running 0 4m
```

```
$ kubectl logs -f po/es-master-2173384742-40g9j
[2017-02-01T09:34:15,118][INFO ][o.e.n.Node ] [] initializing ...
[2017-02-01T09:34:15,538][INFO ][o.e.e.NodeEnvironment ] [5gNfx3_] using [1] data paths, mounts [[/data (/dev/sda9)]], net usable_space [13.7gb], net total_space [15.5gb], spins? [possibly], types [ext4]
[2017-02-01T09:34:15,539][INFO ][o.e.e.NodeEnvironment ] [5gNfx3_] heap size [247.5mb], compressed ordinary object pointers [true]
[2017-02-01T09:34:15,545][INFO ][o.e.n.Node ] node name [5gNfx3_] derived from node ID [5gNfx3_GSrmZiChMKhBgIQ]; set [node.name] to override
[2017-02-01T09:34:15,551][INFO ][o.e.n.Node ] version[5.2.0], pid[13], build[24e05b9/2017-01-24T19:52:35.800Z], OS[Linux/4.8.17-coreos/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_111/25.111-b14]
[2017-02-01T09:34:19,676][INFO ][o.e.p.PluginsService ] [5gNfx3_] loaded module [aggs-matrix-stats]
[2017-02-01T09:34:19,676][INFO ][o.e.p.PluginsService ] [5gNfx3_] loaded module [ingest-common]
[2017-02-01T09:34:19,676][INFO ][o.e.p.PluginsService ] [5gNfx3_] loaded module [lang-expression]
[2017-02-01T09:34:19,677][INFO ][o.e.p.PluginsService ] [5gNfx3_] loaded module [lang-groovy]
[2017-02-01T09:34:19,677][INFO ][o.e.p.PluginsService ] [5gNfx3_] loaded module [lang-mustache]
[2017-02-01T09:34:19,678][INFO ][o.e.p.PluginsService ] [5gNfx3_] loaded module [lang-painless]
[2017-02-01T09:34:19,678][INFO ][o.e.p.PluginsService ] [5gNfx3_] loaded module [percolator]
[2017-02-01T09:34:19,690][INFO ][o.e.p.PluginsService ] [5gNfx3_] loaded module [reindex]
[2017-02-01T09:34:19,691][INFO ][o.e.p.PluginsService ] [5gNfx3_] loaded module [transport-netty3]
[2017-02-01T09:34:19,691][INFO ][o.e.p.PluginsService ] [5gNfx3_] loaded module [transport-netty4]
[2017-02-01T09:34:19,693][INFO ][o.e.p.PluginsService ] [5gNfx3_] no plugins loaded
[2017-02-01T09:34:20,717][WARN ][o.e.d.s.g.GroovyScriptEngineService] [groovy] scripts are deprecated, use [painless] scripts instead
[2017-02-01T09:34:34,927][INFO ][o.e.n.Node ] initialized
[2017-02-01T09:34:34,927][INFO ][o.e.n.Node ] [5gNfx3_] starting ...
[2017-02-01T09:34:35,136][WARN ][i.n.u.i.MacAddressUtil ] Failed to find a usable hardware address from the network interfaces; using random bytes: 15:8d:90:01:27:3f:55:d8
[2017-02-01T09:34:35,271][INFO ][o.e.t.TransportService ] [5gNfx3_] publish_address {10.244.56.2:9300}, bound_addresses {10.244.56.2:9300}
[2017-02-01T09:34:35,287][INFO ][o.e.b.BootstrapChecks ] [5gNfx3_] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
[2017-02-01T09:34:38,548][INFO ][o.e.c.s.ClusterService ] [5gNfx3_] new_master {5gNfx3_}{5gNfx3_GSrmZiChMKhBgIQ}{VV-TQnHBRFCwU_AK_aHKpA}{10.244.56.2}{10.244.56.2:9300}, added {{V4ZOb1u}{V4ZOb1ugSwu_1JjxaihuCw}{NLKOlJN4SNeUWCrfR8Q_MA}{10.244.44.2}{10.244.44.2:9300},}, reason: zen-disco-elected-as-master ([1] nodes joined)[{V4ZOb1u}{V4ZOb1ugSwu_1JjxaihuCw}{NLKOlJN4SNeUWCrfR8Q_MA}{10.244.44.2}{10.244.44.2:9300}]
[2017-02-01T09:34:38,667][INFO ][o.e.n.Node ] [5gNfx3_] started
[2017-02-01T09:34:38,843][INFO ][o.e.g.GatewayService ] [5gNfx3_] recovered [0] indices into cluster_state
[2017-02-01T09:34:40,766][INFO ][o.e.c.s.ClusterService ] [5gNfx3_] added {{s0DFRMx}{s0DFRMxOR9GvMPhdoRj-Og}{sm0_EP7kTBOvkpBNFgpFrA}{10.244.9.2}{10.244.9.2:9300},}, reason: zen-disco-node-join[{s0DFRMx}{s0DFRMxOR9GvMPhdoRj-Og}{sm0_EP7kTBOvkpBNFgpFrA}{10.244.9.2}{10.244.9.2:9300}]
[2017-02-01T09:36:02,176][INFO ][o.e.c.s.ClusterService ] [5gNfx3_] added {{EVWrdUh}{EVWrdUh6SI2vLz5qYCO11g}{TNbwoaOnTFq3q8ZMW2oedw}{10.244.56.3}{10.244.56.3:9300},}, reason: zen-disco-node-join[{EVWrdUh}{EVWrdUh6SI2vLz5qYCO11g}{TNbwoaOnTFq3q8ZMW2oedw}{10.244.56.3}{10.244.56.3:9300}]
[2017-02-01T09:36:28,567][INFO ][o.e.c.s.ClusterService ] [5gNfx3_] added {{9Qyjemc}{9QyjemcWQJKTR5zQoj23Rg}{p0pLUaTtTmCOCleEb2BnOA}{10.244.44.4}{10.244.44.4:9300},}, reason: zen-disco-node-join[{9Qyjemc}{9QyjemcWQJKTR5zQoj23Rg}{p0pLUaTtTmCOCleEb2BnOA}{10.244.44.4}{10.244.44.4:9300}]
[2017-02-01T09:36:29,169][INFO ][o.e.c.s.ClusterService ] [5gNfx3_] added {{civ5bHu}{civ5bHuWTKOzFLyRuBDexQ}{MsOr_2peTkClI-RuE6XjZQ}{10.244.44.3}{10.244.44.3:9300},}, reason: zen-disco-node-join[{civ5bHu}{civ5bHuWTKOzFLyRuBDexQ}{MsOr_2peTkClI-RuE6XjZQ}{10.244.44.3}{10.244.44.3:9300}]
[2017-02-01T09:36:40,953][INFO ][o.e.c.s.ClusterService ] [5gNfx3_] added {{IfNeRsV}{IfNeRsVzQT6Gqds3BLt0Pw}{LnxQsC_xRy60EBXFCaww_Q}{10.244.67.3}{10.244.67.3:9300},}, reason: zen-disco-node-join[{IfNeRsV}{IfNeRsVzQT6Gqds3BLt0Pw}{LnxQsC_xRy60EBXFCaww_Q}{10.244.67.3}{10.244.67.3:9300}]
po/es-client-2651269046-rm5bh 1/1 Running 0 1m
po/es-client-2651269046-v0375 1/1 Running 0 1m
po/es-data-2710897177-3p30t 1/1 Running 0 1m
po/es-data-2710897177-hrzmk 1/1 Running 0 1m
po/es-master-507722841-mfv5l 1/1 Running 0 4m
po/es-master-507722841-nrvnc 1/1 Running 0 4m
po/es-master-507722841-x3nzk 1/1 Running 0 4m
```

```
$ kubectl logs -f po/es-master-507722841-mfv5l
[2017-03-01T21:51:20,984][INFO ][o.e.n.Node ] [] initializing ...
[2017-03-01T21:51:21,333][INFO ][o.e.e.NodeEnvironment ] [3abL8FY] using [1] data paths, mounts [[/data (/dev/sda9)]], net usable_space [13.5gb], net total_space [15.5gb], spins? [possibly], types [ext4]
[2017-03-01T21:51:21,335][INFO ][o.e.e.NodeEnvironment ] [3abL8FY] heap size [247.5mb], compressed ordinary object pointers [true]
[2017-03-01T21:51:21,347][INFO ][o.e.n.Node ] node name [3abL8FY] derived from node ID [3abL8FYRSXecEw16r_FHsg]; set [node.name] to override
[2017-03-01T21:51:21,359][INFO ][o.e.n.Node ] version[5.2.2], pid[13], build[f9d9b74/2017-02-24T17:26:45.835Z], OS[Linux/4.9.9-coreos-r1/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_111/25.111-b14]
[2017-03-01T21:51:34,605][INFO ][o.e.p.PluginsService ] [3abL8FY] loaded module [aggs-matrix-stats]
[2017-03-01T21:51:34,619][INFO ][o.e.p.PluginsService ] [3abL8FY] loaded module [ingest-common]
[2017-03-01T21:51:34,621][INFO ][o.e.p.PluginsService ] [3abL8FY] loaded module [lang-expression]
[2017-03-01T21:51:34,621][INFO ][o.e.p.PluginsService ] [3abL8FY] loaded module [lang-groovy]
[2017-03-01T21:51:34,622][INFO ][o.e.p.PluginsService ] [3abL8FY] loaded module [lang-mustache]
[2017-03-01T21:51:34,622][INFO ][o.e.p.PluginsService ] [3abL8FY] loaded module [lang-painless]
[2017-03-01T21:51:34,622][INFO ][o.e.p.PluginsService ] [3abL8FY] loaded module [percolator]
[2017-03-01T21:51:34,623][INFO ][o.e.p.PluginsService ] [3abL8FY] loaded module [reindex]
[2017-03-01T21:51:34,623][INFO ][o.e.p.PluginsService ] [3abL8FY] loaded module [transport-netty3]
[2017-03-01T21:51:34,623][INFO ][o.e.p.PluginsService ] [3abL8FY] loaded module [transport-netty4]
[2017-03-01T21:51:34,625][INFO ][o.e.p.PluginsService ] [3abL8FY] no plugins loaded
[2017-03-01T21:51:35,426][WARN ][o.e.d.s.g.GroovyScriptEngineService] [groovy] scripts are deprecated, use [painless] scripts instead
[2017-03-01T21:51:45,337][INFO ][o.e.n.Node ] initialized
[2017-03-01T21:51:45,337][INFO ][o.e.n.Node ] [3abL8FY] starting ...
[2017-03-01T21:51:45,676][WARN ][i.n.u.i.MacAddressUtil ] Failed to find a usable hardware address from the network interfaces; using random bytes: 56:f2:bd:6e:4c:b0:b9:bd
[2017-03-01T21:51:45,915][INFO ][o.e.t.TransportService ] [3abL8FY] publish_address {10.244.57.3:9300}, bound_addresses {10.244.57.3:9300}
[2017-03-01T21:51:45,943][INFO ][o.e.b.BootstrapChecks ] [3abL8FY] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
[2017-03-01T21:51:51,084][WARN ][o.e.d.z.UnicastZenPing ] [3abL8FY] timed out after [5s] resolving host [elasticsearch-discovery]
[2017-03-01T21:51:54,131][INFO ][o.e.c.s.ClusterService ] [3abL8FY] new_master {3abL8FY}{3abL8FYRSXecEw16r_FHsg}{4wbiiYaYT5Wy0jcghGRArg}{10.244.57.3}{10.244.57.3:9300}, added {{5UCruB1}{5UCruB10TvGfXjw16NlOPw}{20reg1VUTQeNsu5BMcuiKg}{10.244.4.2}{10.244.4.2:9300},}, reason: zen-disco-elected-as-master ([1] nodes joined)[{5UCruB1}{5UCruB10TvGfXjw16NlOPw}{20reg1VUTQeNsu5BMcuiKg}{10.244.4.2}{10.244.4.2:9300}]
[2017-03-01T21:51:54,206][INFO ][o.e.n.Node ] [3abL8FY] started
[2017-03-01T21:51:54,342][INFO ][o.e.g.GatewayService ] [3abL8FY] recovered [0] indices into cluster_state
[2017-03-01T21:51:54,377][INFO ][o.e.c.s.ClusterService ] [3abL8FY] added {{I9TVnBn}{I9TVnBnCQMK942M_sSN-Bg}{6w7fYzniRgiPg59NHOpw4A}{10.244.100.2}{10.244.100.2:9300},}, reason: zen-disco-node-join[{I9TVnBn}{I9TVnBnCQMK942M_sSN-Bg}{6w7fYzniRgiPg59NHOpw4A}{10.244.100.2}{10.244.100.2:9300}]
[2017-03-01T21:52:57,010][INFO ][o.e.c.s.ClusterService ] [3abL8FY] added {{u3LSDKo}{u3LSDKoFR42ZqL4PeR3Zdw}{aqqAlhCjSIK56sn28DLdOg}{10.244.4.3}{10.244.4.3:9300},}, reason: zen-disco-node-join[{u3LSDKo}{u3LSDKoFR42ZqL4PeR3Zdw}{aqqAlhCjSIK56sn28DLdOg}{10.244.4.3}{10.244.4.3:9300}]
[2017-03-01T21:53:29,438][INFO ][o.e.c.s.ClusterService ] [3abL8FY] added {{ckMzgPT}{ckMzgPTbSC2pLmxj6OuAPw}{mXpSkg-FSs-PRuR2fA6hIw}{10.244.22.2}{10.244.22.2:9300},}, reason: zen-disco-node-join[{ckMzgPT}{ckMzgPTbSC2pLmxj6OuAPw}{mXpSkg-FSs-PRuR2fA6hIw}{10.244.22.2}{10.244.22.2:9300}]
[2017-03-01T21:53:38,118][INFO ][o.e.c.s.ClusterService ] [3abL8FY] added {{Oys65WA}{Oys65WAgT1ydEJ9Nzlxe0g}{gvgZdE9lQTuOewkYfdGBVQ}{10.244.22.3}{10.244.22.3:9300},}, reason: zen-disco-node-join[{Oys65WA}{Oys65WAgT1ydEJ9Nzlxe0g}{gvgZdE9lQTuOewkYfdGBVQ}{10.244.22.3}{10.244.22.3:9300}]
```

As you can assert, the cluster is up and running. Easy, wasn't it?
Expand All @@ -108,29 +108,29 @@ As you can assert, the cluster is up and running. Easy, wasn't it?

```
$ kubectl get svc elasticsearch
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
elasticsearch 10.100.203.98 <pending> 9200:32057/TCP 7m
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
elasticsearch 10.100.18.90 <pending> 9200:30565/TCP 33m
```

From any host on your cluster (that's running `kube-proxy`), run:

```
curl http://10.100.203.98:9200
curl http://10.100.18.90:9200
```

You should see something similar to the following:

```json
{
"name" : "IfNeRsV",
"name" : "gA5RwhW",
"cluster_name" : "myesdb",
"cluster_uuid" : "tLXdVPqCRsax2Hm9VCSFjQ",
"cluster_uuid" : "_na_",
"version" : {
"number" : "5.2.0",
"build_hash" : "24e05b9",
"build_date" : "2017-01-24T19:52:35.800Z",
"number" : "5.2.2",
"build_hash" : "f9d9b74",
"build_date" : "2017-02-24T17:26:45.835Z",
"build_snapshot" : false,
"lucene_version" : "6.4.0"
"lucene_version" : "6.4.1"
},
"tagline" : "You Know, for Search"
}
Expand All @@ -139,7 +139,7 @@ You should see something similar to the following:
Or if you want to see cluster information:

```
curl http://10.100.203.98:9200/_cluster/health?pretty
curl http://10.100.18.90:9200/_cluster/health?pretty
```

You should see something similar to the following:
Expand Down Expand Up @@ -194,7 +194,7 @@ If you want to remove the curator job, just run:
```
kubectl delete cronjob curator
kubectl delete configmap curator-config
```
```

## FAQ
### Why does `NUMBER_OF_MASTERS` differ from number of master-replicas?
Expand Down
2 changes: 1 addition & 1 deletion es-client.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ spec:
add:
- IPC_LOCK
- SYS_RESOURCE
image: quay.io/pires/docker-elasticsearch-kubernetes:5.2.0
image: quay.io/pires/docker-elasticsearch-kubernetes:5.2.2
imagePullPolicy: Always
env:
- name: NAMESPACE
Expand Down
2 changes: 1 addition & 1 deletion es-data.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ spec:
add:
- IPC_LOCK
- SYS_RESOURCE
image: quay.io/pires/docker-elasticsearch-kubernetes:5.2.0
image: quay.io/pires/docker-elasticsearch-kubernetes:5.2.2
imagePullPolicy: Always
env:
- name: NAMESPACE
Expand Down
2 changes: 1 addition & 1 deletion es-master.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ spec:
add:
- IPC_LOCK
- SYS_RESOURCE
image: quay.io/pires/docker-elasticsearch-kubernetes:5.2.0
image: quay.io/pires/docker-elasticsearch-kubernetes:5.2.2
imagePullPolicy: Always
env:
- name: NAMESPACE
Expand Down
2 changes: 1 addition & 1 deletion stateful/es-data-stateful.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ spec:
capabilities:
add:
- IPC_LOCK
image: quay.io/pires/docker-elasticsearch-kubernetes:5.2.0
image: quay.io/pires/docker-elasticsearch-kubernetes:5.2.2
imagePullPolicy: Always
env:
- name: NAMESPACE
Expand Down

0 comments on commit c33b218

Please sign in to comment.