Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Operator tcp input adjustable buffer #256

Merged
merged 8 commits into from
Mar 1, 2021

Conversation

jsirianni
Copy link
Member

@jsirianni jsirianni commented Feb 26, 2021

Description of Changes

Added adjustable buffer size for TCP input operator. Defaults to 1MB, with a max size of 10MB. The default buffer is 64k. https://golang.org/pkg/bufio/#Scanner.Buffer

Added test cases for building the operator.

Resolves #255

Please check that the PR fulfills these requirements

  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)
  • Add a changelog entry (for non-trivial bug fixes / features)
  • CI passes

@djaglowski
Copy link
Member

Log Files Logs / Second CPU Avg (%) CPU Avg Δ (%) Memory Avg (MB) Memory Avg Δ (MB)
1 1000 1.6724412 -0.03448713 121.6347 -3.541893
1 5000 5.913964 -0.068909645 132.15086 -2.2999725
1 10000 11.758862 +0.20689297 142.1603 +6.0841827
1 50000 58.29513 +5.223572 182.19962 +12.970352
1 100000 113.34547 +0.0850296 244.13739 +17.391968
10 100 2.1897936 +1.4781952e-05 126.34011 -0.45703125
10 500 6.6553516 -0.03447342 137.1339 +1.0538788
10 1000 13.172646 -0.67242527 141.54984 +0.13630676
10 5000 60.56173 -2.1987114 184.12137 +3.4229584
10 10000 128.77829 +3.455513 218.76186 -8.93251

@codecov
Copy link

codecov bot commented Feb 26, 2021

Codecov Report

Merging #256 (bd9528e) into master (2de92df) will increase coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #256   +/-   ##
=======================================
  Coverage   71.35%   71.35%           
=======================================
  Files         102      102           
  Lines        5627     5634    +7     
=======================================
+ Hits         4015     4020    +5     
- Misses       1178     1181    +3     
+ Partials      434      433    -1     
Impacted Files Coverage Δ
operator/builtin/input/tcp/tcp.go 68.13% <100.00%> (+2.66%) ⬆️
operator/builtin/output/otlp/otlp.go 61.73% <0.00%> (-3.70%) ⬇️
operator/builtin/output/newrelic/newrelic.go 71.96% <0.00%> (+0.93%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2de92df...bd9528e. Read the comment docs.

@jsirianni jsirianni requested a review from djaglowski February 26, 2021 21:31
@jsirianni jsirianni marked this pull request as ready for review February 26, 2021 21:31
operator/builtin/input/tcp/tcp.go Outdated Show resolved Hide resolved
operator/builtin/input/tcp/tcp.go Outdated Show resolved Hide resolved
@djaglowski
Copy link
Member

Log Files Logs / Second CPU Avg (%) CPU Avg Δ (%) Memory Avg (MB) Memory Avg Δ (MB)
1 1000 1.7586849 +0.0517565 122.53489 -2.6417007
1 5000 6.120864 +0.13799047 132.00215 -2.4486847
1 10000 11.345114 -0.20685482 141.98167 +5.9055634
1 50000 54.55223 +1.4806709 173.87029 +4.6410217
1 100000 108.69713 -4.5633087 229.77734 +3.0319214
10 100 2.3448856 +0.15510678 128.73047 +1.9333267
10 500 6.534658 -0.1551671 136.76334 +0.6833191
10 1000 12.81061 -1.034461 142.13133 +0.71780396
10 5000 62.966854 +0.20641327 185.50323 +4.804825
10 10000 116.387115 -8.935661 223.79163 -3.9027405

@jsirianni jsirianni requested a review from djaglowski March 1, 2021 13:49
Copy link
Member

@djaglowski djaglowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor comment, but LGTM

@djaglowski
Copy link
Member

Log Files Logs / Second CPU Avg (%) CPU Avg Δ (%) Memory Avg (MB) Memory Avg Δ (MB)
1 1000 1.7586473 +0.05171895 119.413925 -5.762665
1 5000 5.3276725 -0.65520096 137.24489 +2.7940521
1 10000 12.448662 +0.8966932 139.49232 +3.416214
1 50000 54.380806 +1.3092461 172.54567 +3.3164062
1 100000 109.709885 -3.5505524 227.29648 +0.5510559
10 100 2.3276963 +0.13791752 132.7931 +5.995964
10 500 7.672656 +0.982831 133.37257 -2.7074432
10 1000 13.552096 -0.29297447 145.1584 +3.744873
10 5000 65.071266 +2.3108253 182.1774 +1.4789886
10 10000 126.74573 +1.4229507 223.28152 -4.412842

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

tcp input buffer is too small for vcenter syslog
2 participants