Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow binding to multiple addresses #13592

Closed
nik9000 opened this issue Sep 15, 2015 · 26 comments
Closed

Allow binding to multiple addresses #13592

nik9000 opened this issue Sep 15, 2015 · 26 comments
Labels
:Distributed Coordination/Network Http and internode communication implementations >enhancement

Comments

@nik9000
Copy link
Member

nik9000 commented Sep 15, 2015

Right now you can bind Elasticsearch to only a single address with either:

network.bind_host: "some_ip"

or

network.host: "some_ip"

It'd be convenient to be able to bind to multiple hosts like so:

network.bind_host: [ "some_ip", "some_other_ip" ]

In particular it'd allow the convenient configuration:

network.bind_host: [ "some_data_center_accessible_ip", "localhost" ]
network.publish_host: "some_data_center_accessible_ip"

Which would allow elasticsearch to respond to requests to localhost.

@nik9000 nik9000 added the :Distributed Coordination/Network Http and internode communication implementations label Sep 15, 2015
@rayward
Copy link

rayward commented Sep 18, 2015

+1

When testing ES 2.0, I had to bind to 0.0.0.0 so that ES was accessible from both within and outside of my developer VM.

This should also apply to the http module.

@rmuir
Copy link
Contributor

rmuir commented Sep 18, 2015

I plan to look into this, the code should easily support it, we just have to deal with the configuration to accept it.

I really really do not like people being forced to bind to all addresses when they really just want maybe two or three or whatever.

rmuir added a commit to rmuir/elasticsearch that referenced this issue Oct 6, 2015
For consistency and simplicity this means that es.network.host/bind_host/publish_host can all take arrays.

Yes, only the "best" address from publish_host will still be used, I do not change that, but its already
the case today (even by default: _local_ !), and its just an internal limitation we can later fix. Instead we
just continue with the same logic of "magic selection of publish host" when users are not explicit, and we
are just clear to the user about what is happening.

I did it this way, because network stuff is already overwhelmed, and i mean really overwhelmed with settings.
We need to make this easy on the user to do the right thing and not make that worse.

Examples:

bin/elasticsearch # defaults
```
[elasticsearch] [2015-10-06 01:15:28,671][WARN ][common.network           ] [Todd Arliss] publish host: [_local_] resolves to multiple addresses, auto-selecting {127.0.0.1} as single publish address
[elasticsearch] [2015-10-06 01:15:28,672][INFO ][transport                ] [Todd Arliss] publish_address {127.0.0.1:9300}, bound_addresses {[fe80::1]:9300}, {[::1]:9300}, {127.0.0.1:9300}
```

bin/elasticsearch -Des.network.host=_lo0_,_en0_ # multiple interfaces
```
[2015-10-06 01:17:08,970][WARN ][common.network           ] [Torso] publish host: [_lo0_, _en0_] resolves to multiple addresses, auto-selecting {192.168.0.19} as single publish address
[2015-10-06 01:17:08,972][INFO ][transport                ] [Torso] publish_address {192.168.0.19:9300}, bound_addresses {[fe80::1]:9300}, {[::1]:9300}, {127.0.0.1:9300}, {[fe80::3e15:c2ff:fee5:d26c]:9300}, {192.168.0.19:9300}
```

bin/elasticsearch -Des.network.host=192.168.0.19,_local_ -Des.network.publish_host=192.168.0.19 # being explicit
```
[2015-10-06 01:19:28,603][INFO ][transport                ] [Thunderbird] publish_address {192.168.0.19:9300}, bound_addresses {[fe80::1]:9300}, {[::1]:9300}, {127.0.0.1:9300}, {192.168.0.19:9300}
```

Closes elastic#13592
rmuir added a commit to rmuir/elasticsearch that referenced this issue Oct 6, 2015
For consistency and simplicity this means that es.network.host/bind_host/publish_host can all take arrays.

Yes, we are still limited to a single publish host, and only the "best" address from publish_host will be used,
I do not change that, but its alread the case today (even by default: _local_ !), and its just an internal
limitation we can later fix. Instead we just continue with the same logic of "magic selection of publish host",
and users can always be explicit, and weare just clear to the user about what is happening.

I did it this way, because network stuff is already overwhelmed, and i mean really overwhelmed with settings.
We need to make this easy on the user to do the right thing and not make that worse.

Examples:

bin/elasticsearch # defaults
```
[elasticsearch] [2015-10-06 01:15:28,671][WARN ][common.network           ] [Todd Arliss] publish host: [_local_] resolves to multiple addresses, auto-selecting {127.0.0.1} as single publish address
[elasticsearch] [2015-10-06 01:15:28,672][INFO ][transport                ] [Todd Arliss] publish_address {127.0.0.1:9300}, bound_addresses {[fe80::1]:9300}, {[::1]:9300}, {127.0.0.1:9300}
```

bin/elasticsearch -Des.network.host=_lo0_,_en0_ # multiple interfaces
```
[2015-10-06 01:17:08,970][WARN ][common.network           ] [Torso] publish host: [_lo0_, _en0_] resolves to multiple addresses, auto-selecting {192.168.0.19} as single publish address
[2015-10-06 01:17:08,972][INFO ][transport                ] [Torso] publish_address {192.168.0.19:9300}, bound_addresses {[fe80::1]:9300}, {[::1]:9300}, {127.0.0.1:9300}, {[fe80::3e15:c2ff:fee5:d26c]:9300}, {192.168.0.19:9300}
```

bin/elasticsearch -Des.network.host=192.168.0.19,_local_ -Des.network.publish_host=192.168.0.19 # being explicit
```
[2015-10-06 01:19:28,603][INFO ][transport                ] [Thunderbird] publish_address {192.168.0.19:9300}, bound_addresses {[fe80::1]:9300}, {[::1]:9300}, {127.0.0.1:9300}, {192.168.0.19:9300}
```

Closes elastic#13592
@rmuir rmuir closed this as completed Oct 24, 2015
@vionemc
Copy link

vionemc commented Jan 29, 2016

Doesn't work on ElasticSearch 2.x

@rmuir
Copy link
Contributor

rmuir commented Jan 29, 2016

has not been released.

@vionemc
Copy link

vionemc commented Jan 29, 2016

Ahh, so that's why. Thanks! Really looking forward to it.

@AseemKumar
Copy link

Has this been fixed? Running 2.3.4 and unable to specify multiple interfaces/addresses.

@jasontedor
Copy link
Member

@AseemKumar Yes, it works fine in 2.3.4:

11:40:45 [jason:~] $ elasticsearch -d --network.bind_host=192.168.1.8,192.168.1.9
11:40:47 [jason:~] $ curl -XGET 192.168.1.8:9200/
{
  "name" : "Mutant X",
  "cluster_name" : "elasticsearch_jason",
  "version" : {
    "number" : "2.3.4",
    "build_hash" : "e455fd0c13dceca8dbbdbb1665d068ae55dabe3f",
    "build_timestamp" : "2016-06-30T11:24:31Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.0"
  },
  "tagline" : "You Know, for Search"
}
11:40:55 [jason:~] $ curl -XGET 192.168.1.9:9200/
{
  "name" : "Mutant X",
  "cluster_name" : "elasticsearch_jason",
  "version" : {
    "number" : "2.3.4",
    "build_hash" : "e455fd0c13dceca8dbbdbb1665d068ae55dabe3f",
    "build_timestamp" : "2016-06-30T11:24:31Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.0"
  },
  "tagline" : "You Know, for Search"
}

@AseemKumar
Copy link

AseemKumar commented Jul 14, 2016

Thanks @jasontedor . I specified the interfaces in my config and it only listens on the localhost

network.host: [_enp0s25_, _local_]

Do you know if only IPs can be provided or interface name as well?

@jasontedor
Copy link
Member

jasontedor commented Jul 14, 2016

@AseemKumar Do not specify as an array, but comma-delimited just as I showed above: network.host: _enp0s25_,_local_.

@clintongormley
Copy link
Contributor

@jasontedor if you can't specify as an array, then that's a bug. we use arrays for a number of other settings. the comma-separated format is really for command line settings

@jasontedor
Copy link
Member

@clintongormley I tested this, it does indeed work as an array, and it does work with interface names. I'm not sure why @AseemKumar reported an issue here.

@AseemKumar Can you share exactly what you entered for network.host, any other relevant settings from your configuration file, as well as the output of ifconfig?

@rmuir
Copy link
Contributor

rmuir commented Jul 15, 2016

also if you run with DEBUG logging it prints a "java ifconfig". This is ifconfig as java sees the world.

@podollb
Copy link

podollb commented Aug 4, 2016

I wrestled with this too (trying various formats of specifying multiple values), but I got it working using the array notation, but I needed to quote the values. So I think @AseemKumar needs to quote the values in his array: network.host: ["_enp0s25_", "_local_"] and then it will work. Otherwise, you could use the more specific configs via something like network.bind_host: ["_enp0s25_", "_local_"] and network.publish_host: _enp0s25_

@niemyjski
Copy link
Contributor

I'm running 5.1.1 and I couldn't get it to bind correctly unless I did network.host: [ '10.2.0.7', '_local_' ].

/sbin/ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0d:3a:10:55:07
          inet addr:10.2.0.7  Bcast:10.2.0.255  Mask:255.255.255.0

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0

lxcbr0    Link encap:Ethernet  HWaddr 00:16:3e:00:00:00
          inet addr:10.0.3.1  Bcast:0.0.0.0  Mask:255.255.255.0

I've never had to do this before in 1.7.5. Is this normal to bind just so you can access from localhost and 10.2.0.7 (local ip)

@jasontedor
Copy link
Member

Yes. Elasticsearch binds to localhost only by default, and otherwise only what you explicitly configure.

@Sojoy
Copy link

Sojoy commented Mar 15, 2017

I just tried out rayward's recommendation to ue 0.0.0.0 to bind to all available addresses and it works on ES 5.2.2

@madchap
Copy link

madchap commented Apr 2, 2017

Just a quick note despite the issue being closed, I am not able to do something like (on 2.4.4). I am running under a coreos-kubernetes multi-vagrant setup.

network.host: [ '_local_' , '_site:ipv4_' ]

Getting an error like, and then the container fails.

Exception in thread "main" BindTransportException[Failed to resolve host null]; nested: UnknownHostException[[ '_local_': invalid IPv6 address];
Likely root cause: java.net.UnknownHostException: [ '_local_': invalid IPv6 address

@jasontedor
Copy link
Member

@madchap I tested this and works fine. Are you sure that you input it correctly? I ask because the exception message indicates that it parsed whatever you input as [ '_local_'.

@madchap
Copy link

madchap commented Apr 2, 2017

@jasontedor I tried to catch that to verify it, but could not really as the container is getting destroyed before I can actually get to what is being held in my env variable.

But I tend to believe you're right -- there must be something odd with the env variable interpolation I am missing. Thanks for your confirmation too. I've put 0.0.0.0 for now as I need to move on onto other stuff.

@pvledoux
Copy link

@madchap Did you could resolved that issue? I have a similar problem, I tried different things with no luck:

-network.host=eth0 #works

but

  • network.host=[eth0,'192.168.100.40','X.X.X.X']

is not working. It tried with single quotes, double quotes around eth0.

@madchap
Copy link

madchap commented Apr 22, 2017

@pvledoux, no I eventually did not after having tried what I think is every possibility. I haven't investigated any further. Maybe something related to the particular docker environment in my case?

@pvledoux
Copy link

@madchap Yes I'm docker too. I will investigate on at side when I have more time.

@SPodjasek
Copy link

@madchap @pvledoux Please try:

-e network.host='_site_,_local_'

it works for me

@lodinv
Copy link

lodinv commented May 7, 2019

v 7.0 network.host not work

@RockNHawk
Copy link

this works for me in elastic 7

@lodinv

network.host: 'site,local'
node.name: "node-1"
#discovery.seed_hosts: ["127.0.0.1", "[::1]"]
cluster.initial_master_nodes: ["node-1"]

@DundiVedantam
Copy link

http.host: 10.91.170.115
http.port: 9200
this works for me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed Coordination/Network Http and internode communication implementations >enhancement
Projects
None yet
Development

Successfully merging a pull request may close this issue.