config: separately configure broker bind/config URIs

[garlick]

Currently the "config file" broker bootstrap method looks for a rank-ordered array of zeromq endpoints:

# tbon-endpoints array is ordered by rank (zeromq URI format)
tbon-endpoints = [
    "tcp://192.168.1.100:8020",  # rank 0
    "tcp://192.168.1.101:8020",  # rank 1
    "tcp://192.168.1.102:8020",  # rank 2
]

A broker finds itself in this list by searching through the tbon-endpoints array until it finds an IP address that matches a local interface. It sets its rank to the array index, and calls zmq_bind() on this URI.

A broker finds its TBON parent in the list by computing its parent rank from its rank and the k-ary tree branching factor, and and looking up the URI by index. It then calls zmq_connect() on this URI.

The form of this URI is limited by what is acceptable to both zmq_bind() and zmq_connect(). For example, hostnames are acceptable to zmq_connect() but only IPv4 addresses are acceptable to zmq_bind(). See http://api.zeromq.org/master:zmq-tcp

A more expressive form could be to allow an array entry to be a (hostname, bind_uri, connect_uri) tuple rather than single URI's, e.g.

hosts = [
    [ "fluke2", "tcp://eno1:8020", "tcp://efluke2:8020" ],
    [ "fluke3", "tcp://eno1:8020", "tcp://efluke3:8020" ],
]

The hostname could be matched directly against gethostname() rather than requiring the more complex (and non portable?) getifaddrs() for the interface match.

A possible (optional) abbreviated form where the bind and connect URI's can be derived from the hostname or or are fixed could be something like:

port = 8020
bind = "tcp://eno1:8020"
connect = "tcp://e%h:8020"
hosts = [
    "fluke2",
    "fluke3"
]

[SteVwonder]

Thanks @garlick for writing this up.

hosts = [
    [ "fluke2", "tcp://eno1:8020", "tcp://efluke2:8020" ],
    [ "fluke3", "tcp://eno1:8020", "tcp://efluke3:8020" ],
]

My only nit with this example is that it isn't obvious which uri is the bind and which is the connect from the file. My suggestion would be:

hosts = [
    { hostname = "fluke2", bind = "tcp://eno1:8020", connect = "tcp://efluke2:8020" },
    { hostname = "fluke3", bind = "tcp://eno1:8020", connect = "tcp://efluke3:8020" },
]

This is more verbose with lots of duplication, but it's more self-documenting as well.

[grondo]

And as @garlick suggested above, the config could have some defaults for "missing" values:

port = 8020
default_bind = "tcp://eno1:%P"
default_connect = "tcp://e%h:%P"
hosts = [
   { hostname = "fluke2" },
   { hostname = "fluke3" },
   { hostname = "fluke4", bind = "tcp://eno3:%P" }
]

[garlick]

Yeah, maybe it's a good idea to use a table for the long form entries, particularly if we want both URI's to be optional.

I'm assuming we could support both table entries and bare string (just host) entries in combination?.

Maybe we could support a simple hostlist form consisting of a hostname with embedded idset?

So e.g.

port = 8020
default_bind = "tcp://eno1:%P"
default_connect = "tcp://e%h:%P"
hosts = [
   { hostname = "fluke[2-4]", bind = "tcp://eno3:%P" },
   "fluke[5-128]",
]