Skip to content
This repository has been archived by the owner on May 25, 2022. It is now read-only.

Change entry.Labels to entry.Attributes #51

Merged
merged 3 commits into from
Mar 8, 2021
Merged

Change entry.Labels to entry.Attributes #51

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
17416fd
Change entry.Labels to entry.Attributes
djaglowski Mar 5, 2021
91101ba
Roll back a few mistaken renamed fields
djaglowski Mar 5, 2021
398bc77
Merge main
djaglowski Mar 8, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## Unreleased

### Changed
- Rename `entry.Label` to `entry.Attribute`

## [0.15.1] - 2020-03-01

### Added
Expand Down
4 changes: 2 additions & 2 deletions docs/operators/file_input.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ The `file_input` operator reads logs from files. It will place the lines read in
| `fingerprint_size` | `1kb` | The number of bytes with which to identify a file. The first bytes in the file are used as the fingerprint. Decreasing this value at any point will cause existing fingerprints to forgotten, meaning that all files will be read from the beginning (one time). |
| `max_log_size` | `1MiB` | The maximum size of a log entry to read before failing. Protects against reading large amounts of data into memory |
| `max_concurrent_files` | 1024 | The maximum number of log files from which logs will be read concurrently. If the number of files matched in the `include` pattern exceeds this number, then files will be processed in batches. One batch will be processed per `poll_interval`. |
| `labels` | {} | A map of `key: value` labels to add to the entry's labels |
| `resource` | {} | A map of `key: value` labels to add to the entry's resource |
| `attributes` | {} | A map of `key: value` pairs to add to the entry's attributes |
| `resource` | {} | A map of `key: value` pairs to add to the entry's resource |

Note that by default, no logs will be read unless the monitored file is actively being written to because `start_at` defaults to `end`.

Expand Down
2 changes: 1 addition & 1 deletion docs/operators/filter.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ The `filter` operator filters incoming entries that match an expression.
```yaml
- type: filter
expr: '$labels.env == "production"'
expr: '$attributes.env == "production"'
output: my_output
```
Expand Down
2 changes: 1 addition & 1 deletion docs/operators/host_metadata.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ The `host_metadata` operator adds hostname and ip to the resource of incoming en

### Example Configurations

#### Add static tags and labels
#### Add static attributes

Configuration:
```yaml
Expand Down
4 changes: 2 additions & 2 deletions docs/operators/journald_input.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ The `journald_input` operator will use the `__REALTIME_TIMESTAMP` field of the j
| `files` | | A list of journal files to read entries from |
| `write_to` | $ | The record [field](/docs/types/field.md) written to when creating a new log entry |
| `start_at` | `end` | At startup, where to start reading logs from the file. Options are `beginning` or `end` |
| `labels` | {} | A map of `key: value` labels to add to the entry's labels |
| `resource` | {} | A map of `key: value` labels to add to the entry's resource |
| `attributes` | {} | A map of `key: value` pairs to add to the entry's attributes |
| `resource` | {} | A map of `key: value` pairs to add to the entry's resource |

### Example Configurations

Expand Down
6 changes: 3 additions & 3 deletions docs/operators/k8s_event_input.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ Kubernetes API, and currently requires that Stanza is running inside a Kubernete
| `discover_namespaces` | `true` | If true, the operator will regularly poll for new namespaces to include |
| `discovery_interval ` | `1m` | The interval at which the operator searches for new namespaces to follow |
| `write_to` | $ | The record [field](/docs/types/field.md) written to when creating a new log entry |
| `labels` | {} | A map of `key: value` labels to add to the entry's labels |
| `resource` | {} | A map of `key: value` labels to add to the entry's resource |
| `attributes` | {} | A map of `key: value` pairs to add to the entry's attributes |
| `resource` | {} | A map of `key: value` pairs to add to the entry's resource |

### Example Configurations

Expand All @@ -30,7 +30,7 @@ Output events:
{
"timestamp": "2020-08-13T17:41:44.581552468Z",
"severity": 0,
"labels": {
"attributes": {
"event_type": "ADDED"
},
"record": {
Expand Down
6 changes: 3 additions & 3 deletions docs/operators/k8s_metadata_decorator.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
## `k8s_metadata_decorator` operator

The `k8s_metadata_decorator` operator adds labels and annotations to the entry using data from the Kubernetes metadata API.
The `k8s_metadata_decorator` operator adds attributes to the entry using data from the Kubernetes metadata API.

### Configuration Fields

Expand All @@ -18,7 +18,7 @@ The `k8s_metadata_decorator` operator adds labels and annotations to the entry u
### Example Configurations


#### Add labels and annotations to a log entry
#### Add attributes to a log entry

Configuration:
```yaml
Expand Down Expand Up @@ -46,7 +46,7 @@ Configuration:
```json
{
"timestamp": "",
"labels": {
"attributes": {
"k8s_ns_annotation/addonmanager.kubernetes.io/mode": "Reconcile",
"k8s_ns_annotation/control-plane": "true",
"k8s_ns_annotation/kubernetes.io/cluster-service": "true",
Expand Down
34 changes: 17 additions & 17 deletions docs/operators/metadata.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,30 +1,30 @@
## `metadata` operator

The `metadata` operator adds labels to incoming entries.
The `metadata` operator adds attributes to incoming entries.

### Configuration Fields

| Field | Default | Description |
| --- | --- | --- |
| `id` | `metadata` | A unique identifier for the operator |
| `output` | Next in pipeline | The connected operator(s) that will receive all outbound entries |
| `labels` | {} | A map of `key: value` labels to add to the entry's labels |
| `resource` | {} | A map of `key: value` labels to add to the entry's resource |
| `on_error` | `send` | The behavior of the operator if it encounters an error. See [on_error](/docs/types/on_error.md) |
| Field | Default | Description |
| --- | --- | --- |
| `id` | `metadata` | A unique identifier for the operator |
| `output` | Next in pipeline | The connected operator(s) that will receive all outbound entries |
| `attributes` | {} | A map of `key: value` pairs to add to the entry's attributes |
| `resource` | {} | A map of `key: value` pairs to add to the entry's resource |
| `on_error` | `send` | The behavior of the operator if it encounters an error. See [on_error](/docs/types/on_error.md) |

Inside the label values, an [expression](/docs/types/expression.md) surrounded by `EXPR()`
will be replaced with the evaluated form of the expression. The entry's record can be accessed
with the `$` variable in the expression so labels can be added dynamically from fields.
with the `$` variable in the expression so attributes can be added dynamically from fields.

### Example Configurations


#### Add static labels and resource
#### Add static attributes and resource

Configuration:
```yaml
- type: metadata
labels:
attributes:
environment: "production"
resource:
cluster: "blue"
Expand All @@ -38,7 +38,7 @@ Configuration:
```json
{
"timestamp": "2020-06-15T11:15:50.475364-04:00",
"labels": {},
"attributes": {},
"record": {
"message": "test"
}
Expand All @@ -51,7 +51,7 @@ Configuration:
```json
{
"timestamp": "2020-06-15T11:15:50.475364-04:00",
"labels": {
"attributes": {
"environment": "production"
},
"resource": {
Expand All @@ -67,13 +67,13 @@ Configuration:
</tr>
</table>

#### Add dynamic tags and labels
#### Add dynamic tags and attributes

Configuration:
```yaml
- type: metadata
output: metadata_receiver
labels:
attributes:
environment: 'EXPR( $.environment == "production" ? "prod" : "dev" )'
```

Expand All @@ -85,7 +85,7 @@ Configuration:
```json
{
"timestamp": "2020-06-15T11:15:50.475364-04:00",
"labels": {},
"attributes": {},
"record": {
"production_location": "us_east",
"environment": "nonproduction"
Expand All @@ -99,7 +99,7 @@ Configuration:
```json
{
"timestamp": "2020-06-15T11:15:50.475364-04:00",
"labels": {
"attributes": {
"environment": "dev"
},
"record": {
Expand Down
10 changes: 5 additions & 5 deletions docs/operators/router.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,11 @@ An entry that does not match any of the routes is dropped and not processed furt

#### Route configuration

| Field | Default | Description |
| --- | --- | --- |
| `output` | required | The connected operator(s) that will receive all outbound entries for this route |
| `expr` | required | An [expression](/docs/types/expression.md) that returns a boolean. The record of the routed entry is available as `$` |
| `labels` | {} | A map of `key: value` labels to add to an entry that matches the route |
| Field | Default | Description |
| --- | --- | --- |
| `output` | required | The connected operator(s) that will receive all outbound entries for this route |
| `expr` | required | An [expression](/docs/types/expression.md) that returns a boolean. The record of the routed entry is available as `$` |
| `attributes` | {} | A map of `key: value` pairs to add to an entry that matches the route |


### Examples
Expand Down
18 changes: 9 additions & 9 deletions docs/operators/syslog_input.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,15 @@ The `syslog_input` operator listens for syslog format logs from UDP/TCP packages

### Configuration Fields

| Field | Default | Description |
| ---------- | ---------------- | ------------------------------------------------------------ |
| `id` | `syslog_input` | A unique identifier for the operator |
| `output` | Next in pipeline | The connected operator(s) that will receive all outbound entries |
| `tcp` | {} | A [tcp_input config](./tcp_input.md#configuration-fields) to defined syslog_parser operator. |
| `udp` | {} | A [udp_input config](./udp_input.md#configuration-fields) to defined syslog_parser operator. |
| `syslog` | required | A [syslog parser config](./syslog_parser.md#configuration-fields) to defined syslog_parser operator. |
| `labels` | {} | A map of `key: value` labels to add to the entry's labels |
| `resource` | {} | A map of `key: value` labels to add to the entry's resource |
| Field | Default | Description |
| ---------- | ---------------- | ------------------------------------------------------------ |
| `id` | `syslog_input` | A unique identifier for the operator |
| `output` | Next in pipeline | The connected operator(s) that will receive all outbound entries |
| `tcp` | {} | A [tcp_input config](./tcp_input.md#configuration-fields) to defined syslog_parser operator. |
| `udp` | {} | A [udp_input config](./udp_input.md#configuration-fields) to defined syslog_parser operator. |
| `syslog` | required | A [syslog parser config](./syslog_parser.md#configuration-fields) to defined syslog_parser operator. |
| `attributes` | {} | A map of `key: value` pairs to add to the entry's attributes |
| `resource` | {} | A map of `key: value` pairs to add to the entry's resource |



Expand Down
4 changes: 2 additions & 2 deletions docs/operators/tcp_input.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ The `tcp_input` operator listens for logs on one or more TCP connections. The op
| `listen_address` | required | A listen address of the form `<ip>:<port>` |
| `tls` | | An optional `TLS` configuration (see the TLS configuration section) |
| `write_to` | $ | The record [field](/docs/types/field.md) written to when creating a new log entry |
| `labels` | {} | A map of `key: value` labels to add to the entry's labels |
| `resource` | {} | A map of `key: value` labels to add to the entry's resource |
| `attributes` | {} | A map of `key: value` pairs to add to the entry's attributes |
| `resource` | {} | A map of `key: value` pairs to add to the entry's resource |

#### TLS Configuration

Expand Down
4 changes: 2 additions & 2 deletions docs/operators/udp_input.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ The `udp_input` operator listens for logs from UDP packets.
| `output` | Next in pipeline | The connected operator(s) that will receive all outbound entries |
| `listen_address` | required | A listen address of the form `<ip>:<port>` |
| `write_to` | $ | The record [field](/docs/types/field.md) written to when creating a new log entry |
| `labels` | {} | A map of `key: value` labels to add to the entry's labels |
| `resource` | {} | A map of `key: value` labels to add to the entry's resource |
| `attributes` | {} | A map of `key: value` pairs to add to the entry's attributes |
| `resource` | {} | A map of `key: value` pairs to add to the entry's resource |

### Example Configurations

Expand Down
4 changes: 2 additions & 2 deletions docs/operators/windows_eventlog_input.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ The `windows_eventlog_input` operator reads logs from the windows event log API.
| `start_at` | `end` | On first startup, where to start reading logs from the API. Options are `beginning` or `end` |
| `poll_interval` | 1s | The interval at which the channel is checked for new log entries. This check begins again after all new records have been read |
| `write_to` | $ | The record [field](/docs/types/field.md) written to when creating a new log entry |
| `labels` | {} | A map of `key: value` labels to add to the entry's labels |
| `resource` | {} | A map of `key: value` labels to add to the entry's resource |
| `attributes` | {} | A map of `key: value` pairs to add to the entry's attributes |
| `resource` | {} | A map of `key: value` pairs to add to the entry's resource |

### Example Configurations

Expand Down
4 changes: 2 additions & 2 deletions docs/types/entry.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Entry is the base representation of log data as it moves through a pipeline. All
| `severity` | The [severity](/docs/types/field.md) of the log. |
| `severity_text` | The original text that was interpreted as a [severity](/docs/types/field.md). |
| `resource` | A map of key/value pairs that describe the resource from which the log originated. |
| `labels` | A map of key/value pairs that provide additional context to the log. This value is often used by a consumer to filter logs. |
| `attributes` | A map of key/value pairs that provide additional context to the log. This value is often used by a consumer to filter logs. |
| `record` | The contents of the log. This value is often modified and restructured in the pipeline. It may be a string, number, or object. |


Expand All @@ -20,7 +20,7 @@ Represented in `json` format, an entry may look like the following:
"resource": {
"uuid": "11112222-3333-4444-5555-666677778888",
},
"labels": {
"attributes": {
"env": "prod",
},
"record": {
Expand Down
4 changes: 2 additions & 2 deletions docs/types/expression.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ For reference documentation of the expression language, see [here](https://githu

Available to the expressions are a few special variables:
- `$record` contains the entry's record
- `$labels` contains the entry's labels
- `$attributes` contains the entry's attributes
- `$resource` contains the entry's resource
- `$timestamp` contains the entry's timestamp
- `env()` is a function that allows you to read environment variables
Expand All @@ -19,6 +19,6 @@ Available to the expressions are a few special variables:

```yaml
- type: metadata
labels:
attributes:
stack: 'EXPR(env("STACK"))'
```
Loading