Commits on Feb 8, 2024

1. Remove metrics from compat package …

   This commit removes the metrics from the compat package
   in favour of the existing logging and the additional tools
   at hand, such as amtool, to validate Alertmanager configurations.
   
   Due to the global nature of the compat package, a consequence
   of config.Load, these metrics have proven to be less useful
   in practice than expected, both in Alertmanager and other projects
   such as Mimir.
   
   There are a number of reasons for this:
   
   1. Because the compat package is global, these metrics cannot be
      reset each time config.Load is called, as in multi-tenant
      projects like Mimir loading a config for one tenant would reset
      the metrics for all tenants. This is also the reason the metrics
      are counters and not gauges.
   
   2. Since the metrics are counters, it is difficult to create
      meaningful dashboards for Alertmanager as, unlike in Mimir,
      configurations are not reloaded at fixed intervals, and as such,
      operators cannot use rate to track configuration changes
      over time.
   
   In Alertmanager, there are much better tools available to validate
   that an Alertmanager configuration is compatible with the UTF-8
   parser, including both the existing logging from Alertmanager
   server and amtool check-config.
   
   In other projects like Mimir, we can track configurations for
   individual tenants using log aggregation and storage systems
   such as Loki. This gives operators far more information than
   what is possible with the metrics, including the timestamp,
   input and ID of tenant configurations that are incompatible
   or have disagreement.
   
   Signed-off-by: George Robinson <[email protected]>

   

   grobinson-grafana committed Feb 8, 2024
   Configuration menu

   • View commit details
   • Copy full SHA for e2e574a
   • Browse repository at this point

   Copy the full SHA

   e2e574a View commit details

   Browse the repository at this point in the history