Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make journald emit map[string]interface{} #38

Merged
merged 1 commit into from
Jul 20, 2020
Merged

Conversation

camdencheek
Copy link
Contributor

@camdencheek camdencheek commented Jul 17, 2020

Description of Changes

It turns out our entry.Field doesn't handle iterating through map[string]string well. We ran into this because the journald plugin emits map[string]string. This PR changes the journald input to be more in line with the other plugins so that it emits a map[string]interface{}.

This seems to be part of a larger recurring issue with the types that are allowed in our pipeline. We should really consider locking down the types that are allowed into the record in the first place. It's impossible to handle every type, and if we don't have any standards, it's unclear what is and is not expected to be handled by each of the downstream plugins.

Please check that the PR fulfills these requirements

  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)
  • Add a changelog entry (for non-trivial bug fixes / features)
  • CI passes

@codecov
Copy link

codecov bot commented Jul 17, 2020

Codecov Report

Merging #38 into master will decrease coverage by 0.18%.
The diff coverage is 55.56%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #38      +/-   ##
==========================================
- Coverage   75.44%   75.27%   -0.18%     
==========================================
  Files          61       61              
  Lines        3718     3724       +6     
==========================================
- Hits         2805     2803       -2     
- Misses        690      696       +6     
- Partials      223      225       +2     
Impacted Files Coverage Δ
plugin/builtin/input/journald.go 55.10% <55.56%> (-1.42%) ⬇️
plugin/builtin/output/file.go 51.22% <0.00%> (-9.76%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 89dbd2e...0e9c0ff. Read the comment docs.

@observIQ observIQ deleted a comment from djaglowski Jul 20, 2020
@observIQ observIQ deleted a comment from djaglowski Jul 20, 2020
@djaglowski
Copy link
Member

Log Files Logs / Second CPU Average (%) Memory Average (MB)
1 1000 3.6379902 28.063173
1 5000 6.4311776 36.749462
1 10000 9.431189 43.473732
1 50000 35.484413 118.47549
1 100000 62.56923 192.38362
10 100 4.1379976 28.379715
10 500 6.9311438 32.389008
10 1000 11.086395 45.661232
10 5000 43.345715 86.875946
10 10000 79.983376 199.24582

@camdencheek camdencheek merged commit 5e80eb6 into master Jul 20, 2020
@camdencheek camdencheek deleted the journald-map branch July 20, 2020 20:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants