Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker default recombine operator is skipped #627

Closed
matthewmodestino opened this issue Jan 7, 2023 · 0 comments · Fixed by #626
Closed

Docker default recombine operator is skipped #627

matthewmodestino opened this issue Jan 7, 2023 · 0 comments · Fixed by #626

Comments

@matthewmodestino
Copy link

matthewmodestino commented Jan 7, 2023

Hey Team!

While troubleshooting why our default docker recombine rule was not reassembling partial docker logs from AKS, it was found that our pipeline definition skips it. We originally added it here #467. We need to update the Docker JSON parser operator to ensure it passes the parsed logs to the recombine rule:

    operators:
        - id: parser-docker
          output: handle_empty_log
          timestamp:
            layout: '%Y-%m-%dT%H:%M:%S.%LZ'
            parse_from: attributes.time
          type: json_parser
        - combine_field: attributes.log
          combine_with: ""
          id: docker-recombine
          is_last_entry: attributes.log endsWith "\n"
          output: handle_empty_log
          source_identifier: attributes["log.file.path"]
          type: recombine
        - field: attributes.log
          id: handle_empty_log
          if: attributes.log == nil
          type: add
          value: ""

We need to update the output value in the parser-docker to emit to the docker-recombine and not directly to handle_empty_log OR insert the docker-recombine AFTER the handle_empty_log.

Either way should solve the issue.

Thanks team!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant