First of the collector libraries for gathering data to insert into HostDB.
Given a datacenter, queries the Openstack API to get data about all of the hosts in each tenant, and sends the data to HostDB.
The library can either be executed in a container (recommended) or on the command line after installing the gem.
The ruby gem is built and installed in a container as part of a build.
The build used to be here: https://builds.pdxfixit.com/gh/hostdb-collector-openstack
The final artifact (the container image) was uploaded to the registry at this location: https://registry.pdxfixit.com/hostdb-collector-openstack
Build the container image from the root directory of the cloned Git repository:
$ make build
Build the gem from the root directory of the cloned Git repository:
$ gem build hostdb_collector_openstack.gemspec
And install the freshly built gem:
$ gem install hostdb_collector_openstack-<VERSION>.gem
Requires a DATACENTER variable to be set, and credential pairs for both the Openstack and HostDB APIs to execute.
See the execution instructions on how to supply the required account information
A YAML configuration file is also required to run the application. The default configuration can be viewed here.
Run the docker container previously built, providing the credentials in an environment file:
$ cat ~/.creds
OS_USERNAME=<Openstack Username>
OS_PASSWORD=<Openstack Password>
HOSTDB_USERNAME=<HostDB Username>
HOSTDB_PASSWORD=<HostDB Password>
$ docker run --rm --env-file ~/.creds -e DATACENTER=va2 openstack-collector
OR run the latest docker container built via the build pipeline (recommended):
$ docker run --rm --env-file ~/.creds -e DATACENTER=va2 registry.pdxfixit.com/hostdb-collector-openstack
The username/password combination can either be set in the following four environment variables prior to execution:
$ export OS_USERNAME=<Openstack Username>
$ export OS_PASSWORD=<Openstack Password>
$ export HOSTDB_USERNAME=<HostDB Username>
$ export HOSTDB_PASSWORD=<HostDB Password>
$ export DATACENTER=va2
OR supplied on the command line using the following flags:
--openstack_username <Openstack Username> --openstack_password <Openstack Password> --hostdb_username <HostDB Username> --hostdb_password <HostDB Password> --datacenter <datacenter>
Invoke the collector via the lone script in the bin directory of the repository clone:
$ ruby bin/hostdb_collector_openstack <optional_parameters>
For either method of execution, setting the environment variable B_DEBUG=1 will enable debug logging.
To test the openstack collector locally, a test instance of HostDB is likely required. Please see that project README for information on how to start an instance.
The collector can output data to a file, instead of posting to HostDB. This can useful for debugging, and is used periodically to refresh the sample data available in the hostdb-server project.
Simply run:
$ make sample_data
Which will create a sample-data folder, and run the container with the environment variable SAMPLE_DATA=true and the newly created folder mapped to the container's filesystem root.
The container will generate JSON files for teach tenant it polls, and will then exit.
When opening firewall rules to the Openstack endpoints, the following ports should be included:
- 5000 (identity auth)
- 8774 (nova, for metadata)