Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[exporter/file] Improving write performance speed #17543

Conversation

MovieStoreGuy
Copy link
Contributor

Description:
Using the library bufio to help improve overall performance with the exporter

Link to tracking Issue:
N/A

Testing:

I'ver included benchmarks to help verify improvements, on my local these are the expected changes:

BenchmarkWriter/buffered-discard-10         	67764800	        17.55 ns/op	      24 B/op	       1 allocs/op
BenchmarkWriter/raw-file-10                 	25841067	        46.40 ns/op	      72 B/op	       2 allocs/op
BenchmarkWriter/buffered-file-10            	74804418	        16.13 ns/op	      24 B/op	       1 allocs/op
BenchmarkWriter/discard-10                  	75218797	        15.92 ns/op	      24 B/op	       1 allocs/op

Documentation:

N/A, completely transparent to the user.

@MovieStoreGuy MovieStoreGuy requested review from a team and mx-psi January 12, 2023 14:08
@github-actions github-actions bot requested a review from atingchen January 12, 2023 14:08
@runforesight
Copy link

runforesight bot commented Jan 12, 2023

Foresight Summary

    
Major Impacts

build-and-test-windows duration(5 seconds) has decreased 42 minutes 27 seconds compared to main branch avg(42 minutes 32 seconds).
View More Details

⭕  changelog workflow has finished in 4 seconds (2 minutes less than main branch avg.) and finished at 17th Jan, 2023.


Job Failed Steps Tests
changelog -     🔗  N/A See Details

⭕  build-and-test-windows workflow has finished in 5 seconds (42 minutes 27 seconds less than main branch avg.) and finished at 17th Jan, 2023.


Job Failed Steps Tests
windows-unittest-matrix -     🔗  N/A See Details
windows-unittest -     🔗  N/A See Details

✅  check-links workflow has finished in 41 seconds (57 seconds less than main branch avg.) and finished at 17th Jan, 2023.


Job Failed Steps Tests
changed files -     🔗  N/A See Details
check-links -     🔗  N/A See Details

✅  tracegen workflow has finished in 55 seconds (1 minute 19 seconds less than main branch avg.) and finished at 17th Jan, 2023.


Job Failed Steps Tests
build-dev -     🔗  N/A See Details
publish-latest -     🔗  N/A See Details
publish-stable -     🔗  N/A See Details

✅  telemetrygen workflow has finished in 58 seconds (1 minute 21 seconds less than main branch avg.) and finished at 17th Jan, 2023.


Job Failed Steps Tests
build-dev -     🔗  N/A See Details
publish-latest -     🔗  N/A See Details
publish-stable -     🔗  N/A See Details

 build-and-test workflow has finished in 16 minutes 37 seconds (31 minutes 9 seconds less than main branch avg.) and finished at 17th Jan, 2023.


Job Failed Steps Tests
unittest-matrix (1.19, internal) N/A  ✅ 662  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, internal) N/A  ✅ 662  ❌ 0  ⏭ 0    🔗 See Details
correctness-metrics N/A  ✅ 2  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, processor) N/A  ✅ 1488  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, extension) N/A  ✅ 528  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, extension) N/A  ✅ 528  ❌ 0  ⏭ 0    🔗 See Details
correctness-traces N/A  ✅ 17  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, processor) N/A  ✅ 1488  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, receiver-0) N/A  ✅ 2565  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, receiver-0) N/A  ✅ 2565  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, exporter) N/A  ✅ 2475  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, exporter) N/A  ✅ 2475  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, other) N/A  ✅ 4464  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, receiver-1) N/A  ✅ 1894  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, other) N/A  ✅ 4464  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, receiver-1) N/A  ✅ 1894  ❌ 0  ⏭ 0    🔗 See Details
integration-tests N/A  ✅ 55  ❌ 0  ⏭ 0    🔗 See Details

✅  load-tests workflow has finished in 7 minutes 46 seconds (6 minutes 26 seconds less than main branch avg.) and finished at 17th Jan, 2023.


Job Failed Steps Tests
loadtest (TestTraceAttributesProcessor) -     🔗  ✅ 3  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestIdleMode) -     🔗  ✅ 1  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestMetric10kDPS|TestMetricsFromFile) -     🔗  ✅ 6  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestTraceNoBackend10kSPS|TestTrace1kSPSWithAttrs) -     🔗  ✅ 8  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestTraceBallast1kSPSWithAttrs|TestTraceBallast1kSPSAddAttrs) -     🔗  ✅ 10  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestMetricResourceProcessor|TestTrace10kSPS) -     🔗  ✅ 12  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestBallastMemory|TestLog10kDPS) -     🔗  ✅ 19  ❌ 0  ⏭ 0    🔗 See Details
setup-environment -     🔗  N/A See Details

✅  prometheus-compliance-tests workflow has finished in 14 minutes 30 seconds (⚠️ 6 minutes 55 seconds more than main branch avg.) and finished at 17th Jan, 2023.


Job Failed Steps Tests
prometheus-compliance-tests -     🔗  ✅ 21  ❌ 0  ⏭ 0    🔗 See Details

🔎 See details on Foresight

*You can configure Foresight comments in your organization settings page.

@MovieStoreGuy MovieStoreGuy force-pushed the msg/improve-fileexporter-performance branch from 8473e2a to 1c5031f Compare January 12, 2023 23:37
@MovieStoreGuy MovieStoreGuy force-pushed the msg/improve-fileexporter-performance branch from 1645526 to 2965fe8 Compare January 16, 2023 00:48
@MovieStoreGuy
Copy link
Contributor Author

The latest benchmark results I have from the update are:

BenchmarkWriter/discard_10485760_bytes-10         	514807416	         2.222 ns/op	       0 B/op	       0 allocs/op
BenchmarkWriter/buffered-discard_10485760_bytes-10         	179711494	         6.596 ns/op	       0 B/op	       0 allocs/op
BenchmarkWriter/raw-file_10485760_bytes-10                 	39634153	        28.55 ns/op	      48 B/op	       1 allocs/op
BenchmarkWriter/buffered-file_10485760_bytes-10            	443848808	         2.717 ns/op	       0 B/op	       0 allocs/op
BenchmarkWriter/discard_104857600_bytes-10                 	539187712	         2.244 ns/op	       0 B/op	       0 allocs/op
BenchmarkWriter/buffered-discard_104857600_bytes-10        	182226061	         6.573 ns/op	       0 B/op	       0 allocs/op
BenchmarkWriter/raw-file_104857600_bytes-10                	37745956	        27.93 ns/op	      48 B/op	       1 allocs/op
BenchmarkWriter/buffered-file_104857600_bytes-10           	448611487	         2.686 ns/op	       0 B/op	       0 allocs/op
BenchmarkWriter/discard_1073741824_bytes-10                	541026325	         2.208 ns/op	       0 B/op	       0 allocs/op
BenchmarkWriter/buffered-discard_1073741824_bytes-10       	181645206	         6.629 ns/op	       0 B/op	       0 allocs/op
BenchmarkWriter/raw-file_1073741824_bytes-10               	39887326	        29.18 ns/op	      48 B/op	       1 allocs/op
BenchmarkWriter/buffered-file_1073741824_bytes-10          	443683308	         2.693 ns/op	       0 B/op	       0 allocs/op
BenchmarkWriter/discard_10737418240_bytes-10               	530066356	         2.216 ns/op	       0 B/op	       0 allocs/op
BenchmarkWriter/buffered-discard_10737418240_bytes-10      	182331368	         6.581 ns/op	       0 B/op	       0 allocs/op
BenchmarkWriter/raw-file_10737418240_bytes-10              	       1	3159734250 ns/op	       0 B/op	       0 allocs/op
BenchmarkWriter/buffered-file_10737418240_bytes-10         	       1	3254270750 ns/op	       0 B/op	       0 allocs/op

Which show an increased performance for small writes (bellow the 10MB) but when it is great than the internal buffer then it will end up as the same performance as the raw file.

@MovieStoreGuy MovieStoreGuy added the Skip Changelog PRs that do not require a CHANGELOG.md entry label Jan 17, 2023
@MovieStoreGuy MovieStoreGuy force-pushed the msg/improve-fileexporter-performance branch from 1ef07d2 to b3b5742 Compare January 17, 2023 09:41
exporter/fileexporter/buffered_writer.go Outdated Show resolved Hide resolved
@MovieStoreGuy MovieStoreGuy force-pushed the msg/improve-fileexporter-performance branch from 5d62cda to 7cec50d Compare January 17, 2023 23:30
@MovieStoreGuy MovieStoreGuy force-pushed the msg/improve-fileexporter-performance branch from 7cec50d to 30476a3 Compare January 17, 2023 23:31
@djaglowski djaglowski changed the title [ File Exporter ] Improving write performance speed [exporter/file] Improving write performance speed Jan 17, 2023
@djaglowski djaglowski merged commit e991cc7 into open-telemetry:main Jan 17, 2023
@plantfansam plantfansam mentioned this pull request Jul 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
exporter/file Skip Changelog PRs that do not require a CHANGELOG.md entry
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants