Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs: Add upgrading/troubleshooting details for repos, config sync, agents #7526

Merged
merged 3 commits into from
Sep 23, 2019
Merged

Docs: Add upgrading/troubleshooting details for repos, config sync, agents #7526

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
25f17c0
Upgrading Docs: Add a repository requirement table to clarify things
Sep 23, 2019
bcf73f6
Upgrading/Troubleshooting Docs: Add details about Zone inclusion and …
Sep 23, 2019
91c1b7d
Upgrading/Troubleshooting Docs: Agent Hosts with Command Endpoint req…
Sep 23, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
144 changes: 144 additions & 0 deletions doc/15-troubleshooting.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1393,6 +1393,56 @@ $ curl -k -s -u root:icinga -H 'Accept: application/json' -X POST 'https://local
```


#### Agent Hosts with Command Endpoint require a Zone <a id="troubleshooting-cluster-command-endpoint-errors-agent-hosts-command-endpoint-zone"></a>

2.11 fixes bugs where agent host checks would never be scheduled on
the master. One requirement is that the checkable host/service
is put into a zone.

By default, the Director puts the agent host in `zones.d/master`
and you're good to go. If you manually manage the configuration,
the config compiler now throws an error with `command_endpoint`
being set but no `zone` defined.

In case you previously managed the configuration outside of `zones.d`,
follow along with the following instructions.

The most convenient way with e.g. managing the objects in `conf.d`
is to move them into the `master` zone.

First, verify the name of your endpoint's zone. The CLI wizards
use `master` by default.

```
vim /etc/icinga2/zones.conf

object Zone "master" {
...
}
```

Then create a new directory in `zones.d` called `master`, if not existing.

```
mkdir -p /etc/icinga2/zones.d/master
```

Now move the directory tree from `conf.d` into the `master` zone.

```
mv conf.d/* /etc/icinga2/zones.d/master/
```

Validate the configuration and reload Icinga.

```
icinga2 daemon -C
systemctl restart icinga2
```

Another method is to specify the `zone` attribute manually, but since
this may lead into other unwanted "not checked" scenarios, we don't
recommend this for your production environment.

### Cluster Troubleshooting Config Sync <a id="troubleshooting-cluster-config-sync"></a>

Expand Down Expand Up @@ -1485,6 +1535,100 @@ In order to solve this problem, remove the mentioned files from `zones.d` and us
of syncing plugin binaries to your satellites and agents.


#### Zones in Zones doesn't work <a id="troubleshooting-cluster-config-zones-in-zones"></a>

The cluster config sync works in the way that configuration
put into `/etc/icinga2/zones.d` only is included when configured
outside in `/etc/icinga2/zones.conf`.

If you for example create a "Zone Inception" with defining the
`satellite` zone in `zones.d/master`, the config compiler does not
re-run and include this zone config recursively from `zones.d/satellite`.

Since v2.11, the config compiler is only including directories where a
zone has been configured. Otherwise it would include renamed old zones,
broken zones, etc. and those long-lasting bugs have been now fixed.

A more concrete example: Masters and Satellites still need to know the Zone hierarchy outside of zones.d synced configuration.

**Doesn't work**

```
vim /etc/icinga2/zones.conf

object Zone "master" {
endpoints = [ "icinga2-master1.localdomain", "icinga2-master2.localdomain" ]
}
```

```
vim /etc/icinga2/zones.d/master/satellite-zones.conf

object Zone "satellite" {
endpoints = [ "icinga2-satellite1.localdomain", "icinga2-satellite1.localdomain" ]
}
```

```
vim /etc/icinga2/zones.d/satellite/satellite-hosts.conf

object Host "agent" { ... }
```

The `agent` host object will never reach the satellite, since the master does not have
the `satellite` zone configured outside of zones.d.


**Works**

Each instance needs to know this, and know about the endpoints first:

```
vim /etc/icinga2/zones.conf

object Endpoint "icinga2-master1.localdomain" { ... }
object Endpoint "icinga2-master2.localdomain" { ... }

object Endpoint "icinga2-satellite1.localdomain" { ... }
object Endpoint "icinga2-satellite2.localdomain" { ... }
```

Then the zone hierarchy as trust and also config sync inclusion is required.

```
vim /etc/icinga2/zones.conf

object Zone "master" {
endpoints = [ "icinga2-master1.localdomain", "icinga2-master2.localdomain" ]
}

object Zone "satellite" {
endpoints = [ "icinga2-satellite1.localdomain", "icinga2-satellite1.localdomain" ]
}
```

Once done, you can start deploying actual monitoring objects into the satellite zone.

```
vim /etc/icinga2/zones.d/satellite/satellite-hosts.conf

object Host "agent" { ... }
```

That's also explained and described in the [documentation](06-distributed-monitoring.md#distributed-monitoring-scenarios-master-satellite-agents).

The thing you can do: For `command_endpoint` agents like inside the Director:
Host -> Agent -> yes, there is no config sync for this zone in place. Therefore
it is valid to just sync their zones via the config sync.

> **Summary**
>
> With using the Director, its cluster zones and agent hosts, you are safe.
>
> Manage the master/satellite instances outside in zones.conf and import
> them via kickstart wizard.


### Cluster Troubleshooting Overdue Check Results <a id="troubleshooting-cluster-check-results"></a>

If your master does not receive check results (or any other events) from the child zones
Expand Down
62 changes: 55 additions & 7 deletions doc/16-upgrading-icinga-2.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,22 @@ The rewrite of our core network stack for cluster and REST API
requires newer Boost versions, specifically >= 1.66. For technical
details, please continue reading in [this issue](https://github.com/Icinga/icinga2/issues/7041).

The package dependencies have been updated for RPM/DEB already.
On platforms where EPEL or Backports cannot satisfy this dependency,
we provide Boost as package on our [package repository](https://packages.icinga.com):
Distribution | Repository providing Boost Dependencies
---------------------|-------------------------------------
RHEL/CentOS 7 | [EPEL repository](02-installation.md#package-repositories-rhel-epel)
RHEL/CentOS 6 x64 | [packages.icinga.com](https://packages.icinga.com)
Fedora | Fedora Upstream
Debian 10 Buster | Debian Upstream
Debian 9 Stretch | [Backports repository](02-installation.md#package-repositories-debian-backports) **New since 2.11**
Debian 8 Jessie | [packages.icinga.com](https://packages.icinga.com)
Ubuntu 18 Bionic | [packages.icinga.com](https://packages.icinga.com)
Ubuntu 16 Xenial | [packages.icinga.com](https://packages.icinga.com)
SLES 15 | SUSE Upstream
SLES 12 | [packages.icinga.com](https://packages.icinga.com) (replaces the SDK repository requirement)

* SLES 12 (this replaces the SDK requirement)
* CentOS 6 x64
* Debian Jessie
* Ubuntu Xenial/Bionic
On platforms where EPEL or Backports cannot satisfy this dependency,
we provide Boost as package on our [package repository](https://packages.icinga.com)
for your convenience.

After upgrade, you may remove the old Boost packages (1.53 or anything above)
if you don't need them anymore.
Expand Down Expand Up @@ -144,6 +152,22 @@ please let us know with an issue on GitHub.

### Cluster <a id="upgrading-to-2-11-cluster"></a>

#### Agent Hosts with Command Endpoint require a Zone <a id="upgrading-to-2-11-cluster-agent-hosts-command-endpoint-zone"></a>

2.11 fixes bugs where agent host checks would never be scheduled on
the master. One definite requirement is that the checkable host/service
is put into a zone.

By default, the Director puts the agent host in `zones.d/master`
and you're good to go. If you manually manage the configuration,
the config compiler now throws an error with `command_endpoint`
being set but no `zone` defined.

The most convenient way with e.g. managing the objects in `conf.d`
is to move them into the `master` zone. Please continue in the
[troubleshooting docs](#troubleshooting-cluster-command-endpoint-errors-agent-hosts-command-endpoint-zone)
for further instructions.

#### Config Sync <a id="upgrading-to-2-11-cluster-config-sync"></a>

2.11 overhauls the cluster config sync in many ways. This includes the
Expand Down Expand Up @@ -186,6 +210,30 @@ and trigger reload loops. In order to prevent such harm in production,
use infrastructure tools such as Foreman, Puppet, Ansible, etc. to install
plugins on the masters, satellites and agents.

##### Config Sync: Zones in Zones <a id="upgrading-to-2-11-cluster-config-sync-zones-in-zones"></a>

The cluster config sync works in the way that configuration
put into `/etc/icinga2/zones.d` only is included when configured
outside in `/etc/icinga2/zones.conf`.

If you for example create a "Zone Inception" with defining the
`satellite` zone in `zones.d/master`, the config compiler does not
re-run and include this zone config recursively from `zones.d/satellite`.

Since v2.11, the config compiler is only including directories where a
zone has been configured. Otherwise it would include renamed old zones,
broken zones, etc. and those long-lasting bugs have been now fixed.

Please consult the [troubleshoot docs](15-troubleshooting.md#troubleshooting-cluster-config-zones-in-zones)
for concrete examples and solutions.

> **Note**
>
> With using the Director, its cluster zones and agent hosts, you are safe.
>
> Manage the master/satellite instances outside in zones.conf and import
> them via kickstart wizard.


#### HA-aware Features <a id="upgrading-to-2-11-cluster-ha-aware-features"></a>

Expand Down