In the cloud world, many hosts appear and vanish. Since we don't want to bother to manage some internal DNS service while keeping its availability, we have been updating /etc/hosts file periodically with cron and AWS API.
There are, however, several problems in that way of updating /etc/hosts:
- It's far from real time
- There are many other components that need to be updated in a likely way; for example, nagios, munin, etc.
It's laborious that we have to write scripts for each purposes and edit crontab.
Serf can solve the problem. It provides us decentralized hosts discovery solution. Once we launch serf agents in each hosts, the cluster itself works like it is managed completely. In addition, it's almost real time.
For Mac OS X uses, please check this document to circumvent a problem in OS X at first.
Launch a serf agent named node1:
$ serf agent -node=node1 -bind=127.0.0.10 -rpc-addr=127.0.0.1:7374 -event-handler=event_handler.pl
Then you'll see etc/hosts to be like below:
$ cat etc/hosts
127.0.0.10 node1
Launch another node named node2:
$ serf agent -node=node2 -bind=127.0.0.11 -rpc-addr=127.0.0.1:7375
Then add node2 into the cluster that currently consists of only node1:
$ serf join -rpc-addr=127.0.0.1:7374 127.1.0.11
As a result,
node2is now also a member of the clustermember-joinevent is emittednode2is added intoetc/hosts
$ cat etc/hosts
127.0.0.10 node1
127.0.0.11 node2
Push Ctrl-C to stop node2. Then member-leave event is propagated to node1 and the handler script is fired.
As a result,
node2is now not a member of the clustermember-leaveevent is emittednode2is removed frometc/hosts
$ cat etc/hosts
127.0.0.10 node1