Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

out_influxdb: documentation for tag prefix stripping #1468

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

out_influxdb: documentation for tag prefix stripping #1468

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
cd45857
out_influxdb: add documentation and example use of strip_prefix
Oct 1, 2024
4bff80f
Add comment about bucket versus measurement naming
Oct 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions pipeline/outputs/influxdb.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ The **influxdb** output plugin, allows to flush your records into a [InfluxDB](h
| Bucket | InfluxDB bucket name where records will be inserted - if specified, `database` is ignored and v2 of API is used | |
| Org | InfluxDB organization name where the bucket is \(v2 only\) | fluent |
| Sequence\_Tag | The name of the tag whose value is incremented for the consecutive simultaneous events. | \_seq |
| Strip\_Prefix | String to be stripped from the front of _tag_ when writing influxdb measurement names | |
| HTTP\_User | Optional username for HTTP Basic Authentication | |
| HTTP\_Passwd | Password for user defined in HTTP\_User | |
| HTTP\_Token | Authentication token used with InfluDB v2 - if specified, both HTTP\_User and HTTP\_Passwd are ignored | |
Expand Down Expand Up @@ -109,6 +110,42 @@ Basic example of `Tags_List_Key` usage:
Tag_Keys level status
```

### Prefix stripping

When collecting data from many inputs into many buckets, it can be helpful to remove a common prefix using _Strip_prefix_.

```python
[INPUT]
Name cpu
Tag cpu.one

[INPUT]
Name cpu
Tag cpu.two

[INPUT]
Name cpu
Tag gpu.one

[INPUT]
Name cpu
Tag gpu.two

[OUTPUT]
Name influxdb
Match cpu*
Bucket cpubucket
Strip_prefix cpu.

[OUTPUT]
Name influxdb
Match gpu*
Bucket gpubucket
Strip_prefix gpu.
```

This will result in the buckets _cpubucket_, _gpubucket_ each containing two measurement streams named _one_, _two_. Without prefix stripping, the measurement names would be _cpu.one_, _cpu.two_ (stored in _cpubucket_) and _gpu.one_, _gpu.two_ (stored in _gpubucket_).

### Testing

Before to start Fluent Bit, make sure the target database exists on InfluxDB, using the above example, we will insert the data into a _fluentbit_ database.
Expand Down