Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GELF output writes subsecond portion of timestamp incorrectly, losing leading zeros #3727

Closed
philomory opened this issue Jul 6, 2021 · 1 comment · Fixed by #3898
Closed

Comments

@philomory
Copy link

Bug Report

Describe the bug
Given a timestamp in fluent-bit of the form e.g. 2021-07-05T12:34:56.012345678, the GELF output will render that to a timestamp as 1625488496.12 rather than 1625488496.012 or 1625488496.01 or similar. Basically, the leading zero in the fractional portion of the timestamp is lost.

To Reproduce
Log message:

{"log":"example","stream":"stdout","time":"2021-07-05T12:34:56.012345678"}

fluent-bit.conf:

[SERVICE]
    Parsers_File       parsers.conf

[INPUT]
    Name               tail
    Tag                example
    Path               /tmp/example.log
    Parser             docker
    exit_on_eof        true
    Read_from_Head     true

[OUTPUT]
    Name                      gelf
    Match                     example
    Compress                  false
    Gelf_short_message_key    log

Steps to reproduce:

  1. Write the example log above to /tmp/example.log
  2. Write the example config above to an appropriate location (e.g. /fluent-bit/etc/fluent-bit.conf, if using the fluent-bit container)
  3. In one shell, run nc -u -l 12201 or nc -u -l -p 12201 (depending on platform) to listen for UDP packets and output them to the shell
  4. In another shell, run fluent-bit -c <path-to-config-file>

Expected behavior
nc should receive the following: {"version":"1.1", "short_message":"example", "_stream":"stdout", "_time":"2021-07-05T12:34:56.012345678", "timestamp":1625488496.012}

Actual behavior
nc receives the following: {"version":"1.1", "short_message":"example", "_stream":"stdout", "_time":"2021-07-05T12:34:56.012345678", "timestamp":1625488496.12}

Your Environment

  • Version used: 1.6.10, 1.7.9
  • Configuration: As above
  • Environment name and version (e.g. Kubernetes? What version?): EKS 1.19, also verified outside of Kubernetes
  • Operating System and version: Using the official fluent-bit docker image, so, distroless
  • Filters and plugins: GELF output

Additional context
The source of the bug is https://github.com/fluent/fluent-bit/blob/master/src/flb_pack_gelf.c#L765-L766, specifically in passing integer numbers of seconds and milliseconds to printf with a format string like %u.%lu. If the number of milliseconds is 13, this will be rendered as e.g. 1.13 rather than 1.013. A correct format string would be more along the lines of %u.%03lu. A pull request will follow shortly.

nokute78 added a commit to nokute78/fluent-bit that referenced this issue Aug 2, 2021
It detects fluent#3727

Signed-off-by: Takahiro Yamashita <[email protected]>
nokute78 added a commit to nokute78/fluent-bit that referenced this issue Aug 2, 2021
nokute78 added a commit to nokute78/fluent-bit that referenced this issue Aug 2, 2021
@nokute78
Copy link
Collaborator

nokute78 commented Aug 2, 2021

Thank you for pointing it.
I sent a patch #3898

edsiper pushed a commit that referenced this issue Aug 2, 2021
It detects #3727

Signed-off-by: Takahiro Yamashita <[email protected]>
edsiper pushed a commit that referenced this issue Aug 2, 2021
gautampunhani pushed a commit to gautampunhani/fluent-bit that referenced this issue Aug 4, 2021
gautampunhani pushed a commit to gautampunhani/fluent-bit that referenced this issue Aug 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants