Introduce node.roles setting #54998

jasontedor · 2020-04-09T03:31:32Z

Today we have individual settings for configuring node roles such as node.data and node.master. Additionally, roles are pluggable and we have used this to introduce roles such as node.ml and node.voting_only. As the number of roles is growing, managing these becomes harder for the user. For example, to create a master-only node, today a user has to= configure:

node.data: false
node.ingest: false
node.remote_cluster_client: false
node.ml: false

at a minimum if they are relying on defaults, but also add:

node.master: true
node.transform: false
node.voting_only: false

If they want to be explicit. This is also challenging in cases where a user wants to have configure a coordinating-only node which requires disabling all roles, a list which we are adding to, requiring the user to keep checking whether a node has acquired any of these roles.

This commit addresses this by adding a list setting node.roles for which a user has explicit control over the list of roles that a node has. If the setting is configured, the node has exactly the roles in the list, and not any additional roles. This means to configure a master-only node, the setting is merely 'node.roles: [master]', and to configure a coordinating-only node, the setting is merely: 'node.roles: []'.

With this change we deprecate the existing 'node.*' settings such as 'node.data'.

Today we have individual settings for configuring node roles such as node.data and node.master. Additionally, roles are pluggable and we have used this to introduce roles such as node.ml and node.voting_only. As the number of roles is growing, managing these becomes harder for the user. For example, to create a master-only node, today a user has to configure: - node.data: false - node.ingest: false - node.remote_cluster_client: false - node.ml: false at a minimum if they are relying on defaults, but also add: - node.master: true - node.transform: false - node.voting_only: false If they want to be explicit. This is also challenging in cases where a user wants to have configure a coordinating-only node which requires disabling all roles, a list which we are adding to, requiring the user to keep checking whether a node has acquired any of these roles. This commit addresses this by adding a list setting node.roles for which a user has explicit control over the list of roles that a node has. If the setting is configured, the node has exactly the roles in the list, and not any additional roles. This means to configure a master-only node, the setting is merely 'node.roles: [master]', and to configure a coordinating-only node, the setting is merely: 'node.roles: []'. With this change we deprecate the existing 'node.*' settings such as 'node.data'.

elasticmachine · 2020-04-09T03:31:34Z

Pinging @elastic/es-core-infra (:Core/Infra/Core)

server/src/main/java/org/elasticsearch/node/NodeRoleSettings.java

server/src/test/java/org/elasticsearch/cluster/node/DiscoveryNodeTests.java

server/src/test/java/org/elasticsearch/indices/mapping/DedicatedMasterGetFieldMappingIT.java

test/framework/src/main/java/org/elasticsearch/test/InternalTestCluster.java

pugnascotia · 2020-04-09T13:33:04Z

test/framework/src/main/java/org/elasticsearch/test/NodeRoles.java

+    }
+
+    public static Settings noRoles(final Settings settings) {
+        return Settings.builder().put(settings).putList(NodeRoleSettings.NODE_ROLES_SETTING.getKey(), List.of()).build();


Would the intention be even clearer here if we used Collections.emptyList?

Suggested change

return Settings.builder().put(settings).putList(NodeRoleSettings.NODE_ROLES_SETTING.getKey(), List.of()).build();

return Settings.builder().put(settings).putList(NodeRoleSettings.NODE_ROLES_SETTING.getKey(), emptyList()).build();

...k/plugin/ml/src/test/java/org/elasticsearch/xpack/ml/integration/BasicDistributedJobsIT.java

...k/plugin/ml/src/test/java/org/elasticsearch/xpack/ml/integration/MlDistributedFailureIT.java

This commit update Elasticsearch chart to use the new node.roles settings introduced in elastic/elasticsearch#54998.

* [elasticsearch] use new node.roles settings This commit update Elasticsearch chart to use the new node.roles settings introduced in elastic/elasticsearch#54998. * [elasticsearch] update doc * [elasticsearch] update examples * add link to roles doc * add workaround for coordinating node * update roles list * fixup! update roles list * remove data_frozen from default roles this is needed because `data_frozen` role doesn't exist in Elasticsearch < 7.12.0 * fixup! remove data_frozen from default roles

This commit update Elasticsearch chart to use the new node.roles settings introduced in elastic/elasticsearch#54998.

* [elasticsearch] use new node.roles settings This commit update Elasticsearch chart to use the new node.roles settings introduced in elastic/elasticsearch#54998. * [elasticsearch] update doc * [elasticsearch] update examples * add link to roles doc * add workaround for coordinating node * update roles list * fixup! update roles list * remove data_frozen from default roles this is needed because `data_frozen` role doesn't exist in Elasticsearch < 7.12.0 * fixup! remove data_frozen from default roles

This commit update Elasticsearch chart to use the new node.roles settings introduced in elastic/elasticsearch#54998.

* [elasticsearch] use new node.roles settings This commit update Elasticsearch chart to use the new node.roles settings introduced in elastic/elasticsearch#54998. * [elasticsearch] update doc * [elasticsearch] update examples * add link to roles doc * add workaround for coordinating node * update roles list * fixup! update roles list * remove data_frozen from default roles this is needed because `data_frozen` role doesn't exist in Elasticsearch < 7.12.0 * fixup! remove data_frozen from default roles

This commit update Elasticsearch chart to use the new node.roles settings introduced in elastic/elasticsearch#54998.

* [elasticsearch] use new node.roles settings This commit update Elasticsearch chart to use the new node.roles settings introduced in elastic/elasticsearch#54998. * [elasticsearch] update doc * [elasticsearch] update examples * add link to roles doc * add workaround for coordinating node * update roles list * fixup! update roles list * remove data_frozen from default roles this is needed because `data_frozen` role doesn't exist in Elasticsearch < 7.12.0 * fixup! remove data_frozen from default roles

This commit update Elasticsearch chart to use the new node.roles settings introduced in elastic/elasticsearch#54998.

We deprecated several legacy node role settings in 7.9 with PR #54998. However, we didn't add a related item to the 7.9 breaking changes docs. This adds the missing item. Relates to #71163.

…77766) We deprecated several legacy node role settings in 7.9 with PR #54998. However, we didn't add a related item to the 7.9 breaking changes docs. This adds the missing item. Relates to #71163.

…77768) We deprecated several legacy node role settings in 7.9 with PR #54998. However, we didn't add a related item to the 7.9 breaking changes docs. This adds the missing item. Relates to #71163.

…77767) We deprecated several legacy node role settings in 7.9 with PR #54998. However, we didn't add a related item to the 7.9 breaking changes docs. This adds the missing item. Relates to #71163.

…77771) We deprecated several legacy node role settings in 7.9 with PR #54998. However, we didn't add a related item to the 7.9 breaking changes docs. This adds the missing item. Relates to #71163.

…77770) We deprecated several legacy node role settings in 7.9 with PR #54998. However, we didn't add a related item to the 7.9 breaking changes docs. This adds the missing item. Relates to #71163.

…77769) We deprecated several legacy node role settings in 7.9 with PR #54998. However, we didn't add a related item to the 7.9 breaking changes docs. This adds the missing item. Relates to #71163.

…77772) We deprecated several legacy node role settings in 7.9 with PR #54998. However, we didn't add a related item to the 7.9 breaking changes docs. This adds the missing item. Relates to #71163.

* [elasticsearch] use new node.roles settings This commit update Elasticsearch chart to use the new node.roles settings introduced in elastic/elasticsearch#54998. * [elasticsearch] update doc * [elasticsearch] update examples * add link to roles doc * add workaround for coordinating node * update roles list * fixup! update roles list * remove data_frozen from default roles this is needed because `data_frozen` role doesn't exist in Elasticsearch < 7.12.0 * fixup! remove data_frozen from default roles

This commit update Elasticsearch chart to use the new node.roles settings introduced in elastic/elasticsearch#54998.

* [elasticsearch] use new node.roles settings This commit update Elasticsearch chart to use the new node.roles settings introduced in elastic/elasticsearch#54998. * [elasticsearch] update doc * [elasticsearch] update examples * add link to roles doc * add workaround for coordinating node * update roles list * fixup! update roles list * remove data_frozen from default roles this is needed because `data_frozen` role doesn't exist in Elasticsearch < 7.12.0 * fixup! remove data_frozen from default roles

This commit update Elasticsearch chart to use the new node.roles settings introduced in elastic/elasticsearch#54998.

jasontedor added >enhancement :Core/Infra/Core Core issues without another label v8.0.0 v7.8.0 labels Apr 9, 2020

jasontedor requested a review from rjernst April 9, 2020 03:31

jasontedor marked this pull request as draft April 9, 2020 11:06

jasontedor added 2 commits April 9, 2020 07:09

Checkstyle

254c573

Fix test

eaf2b34