Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Auto discover certain coordinator nodes only. #28831

Closed
linker-c opened this issue Feb 26, 2018 · 4 comments
Closed

Feature Request: Auto discover certain coordinator nodes only. #28831

linker-c opened this issue Feb 26, 2018 · 4 comments
Labels
:Core/Infra/Core Core issues without another label >feature help wanted adoptme

Comments

@linker-c
Copy link

linker-c commented Feb 26, 2018

Describe the feature:
I have setup my ES cluster to write to ingest only nodes, and read from coordinator nodes that's not master nor data nodes.
The problem is if I add more coordinator "client" nodes (not master, not ingest, not data) to server query spikes, there's no way for the client to auto discover those nodes without including either master or data nodes. The workaround I have now is to make those new nodes to be ingest nodes as well.
So I just auto discover ingest nodes for read, but then I won't be able to split write & read nodes.

The only way to discover nodes via REST afaik is to use /_nodes/xyz, where xyz is a comma separated "node roles". Maybe there's another way to do this? Or maybe splitting read from write like what I'm doing is overkill?

@nik9000 nik9000 added >feature :Core/Infra/Core Core issues without another label discuss labels Feb 26, 2018
@nik9000
Copy link
Member

nik9000 commented Feb 26, 2018

I think you could give your coordinating nodes an attribute. Like coordinating:true. See the docs for allocation awareness for how. Then you could use something like _nodes/coordinating:true (documentation) for detection.

I'll mark this as discuss because I think it might be nice to be able to do have more expressive syntax in this API. That, or we can document the node attributes trick explicitly somewhere.

@ywelsch
Copy link
Contributor

ywelsch commented Mar 9, 2018

We discussed this in FixitFriday and would like to add coordinating_only as a filtering option for resolving node ids (affecting APIs such as _nodes). In the meanwhile you can use the workaround suggested by @nik9000.

@ywelsch ywelsch added help wanted adoptme and removed discuss labels Mar 9, 2018
@ywelsch
Copy link
Contributor

ywelsch commented Mar 9, 2018

You can also negate node roles, e.g. master:false. This requires a little hack, though, namely to use the wildcard * as first parameter in the request, so that all nodes are initially selected, and the master-eligible then subtracted, ...

The following should achieve what you're looking for:

/_nodes/*,master:false,data:false,ingest:false

The adoptme here is to simplify the API so that the "wildcard" hack is not needed.

@PnPie
Copy link
Contributor

PnPie commented May 1, 2018

Hi @ywelsch @nik9000,
I made a PR for this.

DaveCTurner pushed a commit that referenced this issue May 9, 2018
Today we can execute cluster API actions on only master, data or ingest nodes
using the `master:true`, `data:true` and `ingest:true` filters, but it is not
so easy to select coordinating-only nodes (i.e. those nodes that are neither
master nor data nor ingest nodes). This change fixes this by adding support for
a `coordinating_only` filter such that `coordinating_only:true` adds all
coordinating-only nodes to the set of selected nodes, and 
`coordinating_only:false` deletes them.

Resolves #28831.
DaveCTurner pushed a commit that referenced this issue May 9, 2018
Today we can execute cluster API actions on only master, data or ingest nodes
using the `master:true`, `data:true` and `ingest:true` filters, but it is not
so easy to select coordinating-only nodes (i.e. those nodes that are neither
master nor data nor ingest nodes). This change fixes this by adding support for
a `coordinating_only` filter such that `coordinating_only:true` adds all
coordinating-only nodes to the set of selected nodes, and 
`coordinating_only:false` deletes them.

Resolves #28831.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Core/Infra/Core Core issues without another label >feature help wanted adoptme
Projects
None yet
Development

No branches or pull requests

4 participants