Skip to content

Commit

Permalink
address review comments
Browse files Browse the repository at this point in the history
Signed-off-by: ChrsMark <[email protected]>
  • Loading branch information
ChrsMark committed May 16, 2024
1 parent 47528d1 commit 24a1f28
Showing 1 changed file with 15 additions and 17 deletions.
32 changes: 15 additions & 17 deletions content/en/blog/2024/otel-collector-container-log-parser/index.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
---
title: Introducing the new container log parser for OpenTelemetry Collector
linkTitle: OTel Collector container log parser
linkTitle: Collector container log parser
date: 2024-05-16
author: '[Christos Markou](https://github.com/ChrsMark) (Elastic)'
cSpell:ignore: Christos containerd Filelog filelog Jaglowski kube Markou
---

Filelog receiver is one of the most commonly used components of the
OpenTelemetry Collector, as indicated by the most recent
[Filelog receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filelogreceiver)
is one of the most commonly used components of the
[OpenTelemetry Collector](/docs/collector), as indicated by the most recent
[survey](/blog/2024/otel-collector-survey/#otel-components-usage). According to
the same survey, it's unsurprising that
[Kubernetes is the leading platform for Collector deployment (80.6%)](/blog/2024/otel-collector-survey/#deployment-scale-and-environment).
Expand Down Expand Up @@ -41,30 +42,27 @@ in container log parsing.
First of all we need to quickly recall the different container log formats that
we can meet out there:

### Docker container logs

`{"log":"INFO: This is a docker log line","stream":"stdout","time":"2024-03-30T08:31:20.545192187Z"}`

### cri-o logs

`2024-04-13T07:59:37.505201169-05:00 stdout F This is a cri-o log line!`

### Containerd logs

`2024-04-22T10:27:25.813799277Z stdout F This is an awesome containerd log line!`
- Docker container logs:
`{"log":"INFO: This is a docker log line","stream":"stdout","time":"2024-03-30T08:31:20.545192187Z"}`
- cri-o logs:
`2024-04-13T07:59:37.505201169-05:00 stdout F This is a cri-o log line!`
- Containerd logs:
`2024-04-22T10:27:25.813799277Z stdout F This is an awesome containerd log line!`

We can notice that cri-o and containerd log formats are quite similar (both
follow the CRI logging format) but with a small difference in the timestamp
format.

Consequently, in order to properly handle these 3 different formats we need 3
different routes of stanza operators as we can see in the
different routes of
[stanza](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/stanza)
operators as we can see in the
[container parser operator issue](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/31959).

In addition, the CRI format can provide partial logs which we would like to
combine them into one at first place:

```console
```text
2024-04-06T00:17:10.113242941Z stdout P This is a very very long line th
2024-04-06T00:17:10.113242941Z stdout P at is really really long and spa
2024-04-06T00:17:10.113242941Z stdout F ns across multiple log entries
Expand Down Expand Up @@ -105,7 +103,7 @@ receivers:
```
That configuration is more than enough to properly parse the log line and
extract all the useful K8s metadata.
extract all the useful Kubernetes metadata.
A log line
`{"log":"INFO: This is a docker log line","stream":"stdout","time":"2024-03-30T08:31:20.545192187Z"}`
Expand Down

0 comments on commit 24a1f28

Please sign in to comment.