Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pkg/stanza/fileconsumer] Reduce noise in benchmark #31516

Merged

Conversation

djaglowski
Copy link
Member

@djaglowski djaglowski commented Feb 29, 2024

This PR reworks the end-to-end benchmark in pkg/stanza/fileconsumer.

The primary goal is to surface a more accurate representation of performance within the benchmark. Several factors were leading to artificial delays which I believe may have drowned out actual impacts. Three significant sources of noise were mitigated:

  1. Every log read was passed to a channel. This caused us to measure lot of time waiting for channels. Now, we only signal when the file is fully consumed.
  2. Every line was generated on the fly. Now we generate a set of lines ahead of time and write these over and over.
  3. Each line was written to file independently. Now we group the pre-generated lines write them all at once. We also write longer lines so that much more content is written per call.

The overall result is a substantially clearer CPU profile and a slightly clearer memory profile.

CPU Profile Before
image

CPU Profile After
image

Memory Profile Before
image

Memory Profile After
image

Additionally, this adds a new benchmark to cover a scenario with many (100) files. Finally, it reduces the time to run the benchmark from ~30s to ~20s.

@djaglowski djaglowski force-pushed the pkg-stanza-fileconsumer-bench-update branch 2 times, most recently from 18d1d5d to 2764c42 Compare March 5, 2024 17:46
@djaglowski djaglowski added the Skip Changelog PRs that do not require a CHANGELOG.md entry label Mar 5, 2024
@djaglowski djaglowski force-pushed the pkg-stanza-fileconsumer-bench-update branch from 2764c42 to af235c9 Compare March 5, 2024 17:51
@djaglowski djaglowski changed the title [pkg/stanza] Reduce noise in benchmark [pkg/stanza/fileconsumer] Reduce noise in benchmark Mar 5, 2024
@djaglowski djaglowski force-pushed the pkg-stanza-fileconsumer-bench-update branch from af235c9 to 38e5a03 Compare March 5, 2024 18:02
@djaglowski djaglowski marked this pull request as ready for review March 5, 2024 18:14
@djaglowski djaglowski requested review from a team and Aneurysm9 March 5, 2024 18:14
@djaglowski djaglowski force-pushed the pkg-stanza-fileconsumer-bench-update branch from 38e5a03 to 03d2a4f Compare March 12, 2024 19:18
Copy link
Contributor

This PR was marked stale due to lack of activity. It will be closed in 14 days.

@github-actions github-actions bot added the Stale label Mar 27, 2024
@djaglowski
Copy link
Member Author

I think this is ready for another review @crobert-1

@djaglowski djaglowski added Skip Changelog PRs that do not require a CHANGELOG.md entry and removed Stale Skip Changelog PRs that do not require a CHANGELOG.md entry labels Mar 27, 2024
@djaglowski djaglowski force-pushed the pkg-stanza-fileconsumer-bench-update branch from 03d2a4f to a71d7cc Compare March 27, 2024 22:08
@djaglowski djaglowski force-pushed the pkg-stanza-fileconsumer-bench-update branch from a71d7cc to 5a9555e Compare March 28, 2024 17:52
@djaglowski djaglowski merged commit b8efeeb into open-telemetry:main Mar 28, 2024
142 checks passed
@djaglowski djaglowski deleted the pkg-stanza-fileconsumer-bench-update branch March 28, 2024 18:32
@github-actions github-actions bot added this to the next release milestone Mar 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pkg/stanza Skip Changelog PRs that do not require a CHANGELOG.md entry
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants