Skip to content

Commit

Permalink
fixes ruler docs & includes ruler configs in cmd/configs + docker img (
Browse files Browse the repository at this point in the history
  • Loading branch information
owen-d authored Sep 22, 2020
1 parent 6f8bfe0 commit 0d64d2d
Show file tree
Hide file tree
Showing 5 changed files with 67 additions and 16 deletions.
3 changes: 2 additions & 1 deletion cmd/loki/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ COPY cmd/loki/loki-docker-config.yaml /etc/loki/local-config.yaml

RUN addgroup -g 10001 -S loki && \
adduser -u 10001 -S loki -G loki
RUN mkdir -p /loki && \
RUN mkdir -p /loki/rules && \
mkdir -p /loki/tmprules && \
chown -R loki:loki /etc/loki /loki

# See https://github.com/grafana/loki/issues/1928
Expand Down
12 changes: 12 additions & 0 deletions cmd/loki/loki-docker-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,15 @@ chunk_store_config:
table_manager:
retention_deletes_enabled: false
retention_period: 0s

ruler:
storage:
type: local
local:
directory: /loki/rules
rule_path: /loki/tmprules
alertmanager_url: http://localhost
ring:
kvstore:
store: inmemory
enable_api: true
12 changes: 12 additions & 0 deletions cmd/loki/loki-local-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,15 @@ chunk_store_config:
table_manager:
retention_deletes_enabled: false
retention_period: 0s

ruler:
storage:
type: local
local:
directory: /tmp/rules
rule_path: /tmp/scratch
alertmanager_url: http://localhost
ring:
kvstore:
store: inmemory
enable_api: true
29 changes: 23 additions & 6 deletions docs/sources/alerting/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,23 @@ weight: 700

Loki includes a component called the Ruler, adapted from our upstream project, Cortex. The Ruler is responsible for continually evaluating a set of configurable queries and then alerting when certain conditions happen, e.g. a high percentage of error logs.

First, ensure the Ruler component is enabled. The following is a basic configuration which loads rules from configuration files (it requires `/tmp/rules` and `/tmp/scratch` exist):

```yaml
ruler:
storage:
type: local
local:
directory: /tmp/rules
rule_path: /tmp/scratch
alertmanager_url: http://localhost
ring:
kvstore:
store: inmemory
enable_api: true

```

## Prometheus Compatible

When running the Ruler (which runs by default in the single binary), Loki accepts rules files and then schedules them for continual evaluation. These are _Prometheus compatible_! This means the rules file has the same structure as in [Prometheus](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/), with the exception that the rules specified are in LogQL.
Expand Down Expand Up @@ -80,10 +97,10 @@ groups:
rules:
- alert: HighPercentageError
expr: |
sum(rate({app="foo", env="production"} |= "error" [5m])) by (job)
/
sum(rate({app="foo", env="production"}[5m])) by (job)
> 0.05
sum(rate({app="foo", env="production"} |= "error" [5m])) by (job)
/
sum(rate({app="foo", env="production"}[5m])) by (job)
> 0.05
for: 10m
labels:
severity: page
Expand Down Expand Up @@ -208,7 +225,7 @@ One option to scale the Ruler is by scaling it horizontally. However, with multi

The possible configurations are listed fully in the configuration [docs](https://grafana.com/docs/loki/latest/configuration/), but in order to shard rules across multiple Rulers, the rules API must be enabled via flag (`-experimental.Ruler.enable-api`) or config file parameter. Secondly, the Ruler requires it's own ring be configured. From there the Rulers will shard and handle the division of rules automatically. Unlike ingesters, Rulers do not hand over responsibility: all rules are re-sharded randomly every time a Ruler is added to or removed from the ring.

A full Ruler config example is:
A full sharding-enabled Ruler example is:

```yaml
Ruler:
Expand Down Expand Up @@ -242,7 +259,7 @@ A typical local configuration might look something like:
With the above configuration, the Ruler would expect the following layout:
```
/tmp/loki/rules/<tenant id>/rules1.yaml
/rules2.yaml
/rules2.yaml
```
Yaml files are expected to be in the [Prometheus format](#Prometheus_Compatible) but include LogQL expressions as specified in the beginning of this doc.

Expand Down
27 changes: 18 additions & 9 deletions docs/sources/api/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ The HTTP API includes the following endpoints:
- [`POST /api/prom/rules/{namespace}`](#set-rule-group)
- [`DELETE /api/prom/rules/{namespace}/{groupName}`](#delete-rule-group)
- [`DELETE /api/prom/rules/{namespace}`](#delete-namespace)
- [`GET /prometheus/api/v1/rules`](#list-rules)
- [`GET /prometheus/api/v1/alerts`](#list-alerts)

## Microservices mode
Expand Down Expand Up @@ -112,20 +113,19 @@ The API endpoints starting with `/loki/` are [Prometheus API-compatible](https:/
These endpoints are exposed by the ruler:

- [`GET /ruler/ring`](#ruler-ring-status)
- [`GET /api/v1/rules`](#list-rules)
- [`GET /api/v1/rules`](#list-rule-groups)
- [`GET /api/v1/rules/{namespace}`](#get-rule-groups-by-namespace)
- [`GET /api/v1/rules/{namespace}/{groupName}`](#get-rule-group)
- [`POST /api/v1/rules/{namespace}`](#set-rule-group)
- [`DELETE /api/v1/rules/{namespace}/{groupName}`](#delete-rule-group)
- [`DELETE /api/v1/rules/{namespace}`](#delete-namespace)
- [`GET /api/prom/rules`](#list-rules)
- [`GET /loki/api/v1/rules`](#list-rule-groups)
- [`GET /loki/api/v1/rules/{namespace}`](#get-rule-groups-by-namespace)
- [`GET /loki/api/v1/rules/{namespace}/{groupName}`](#get-rule-group)
- [`POST /loki/api/v1/rules/{namespace}`](#set-rule-group)
- [`DELETE /loki/api/v1/rules/{namespace}/{groupName}`](#delete-rule-group)
- [`DELETE /loki/api/v1/rules/{namespace}`](#delete-namespace)
- [`GET /api/prom/rules`](#list-rule-groups)
- [`GET /api/prom/rules/{namespace}`](#get-rule-groups-by-namespace)
- [`GET /api/prom/rules/{namespace}/{groupName}`](#get-rule-group)
- [`POST /api/prom/rules/{namespace}`](#set-rule-group)
- [`DELETE /api/prom/rules/{namespace}/{groupName}`](#delete-rule-group)
- [`DELETE /api/prom/rules/{namespace}`](#delete-namespace)
- [`GET /prometheus/api/v1/rules`](#list-rules)
- [`GET /prometheus/api/v1/alerts`](#list-alerts)

A [list of clients](../clients) can be found in the clients documentation.
Expand Down Expand Up @@ -1114,8 +1114,17 @@ Deletes all the rule groups in a namespace (including the namespace itself). Thi

_This experimental endpoint is disabled by default and can be enabled via the `-experimental.ruler.enable-api` CLI flag (or its respective YAML config option)._

_Requires [authentication](#authentication)._
### List rules

```
GET /loki/api/v1/rules
```

Prometheus-compatible rules endpoint to list alerting and recording rules that are currently loaded.

For more information, refer to the [Prometheus rules](https://prometheus.io/docs/prometheus/latest/querying/api/#rules) documentation.

_This experimental endpoint is disabled by default and can be enabled via the `-experimental.ruler.enable-api` CLI flag (or its respective YAML config option)._

### List alerts

Expand Down

0 comments on commit 0d64d2d

Please sign in to comment.