Documentation of performance test 1.3 #1309
Conversation
Signed-off-by: Taylor Gray <[email protected]>
performance-test/results/results.md
Outdated
|
|
||
|  | ||
|
|
||
| Comparing the performance of the latest release of Data Prepper 1.3 against Logstash 7.13.2, the test was configured to simulate 10 clients to send requests as frequently as possible. Each request was contained a batch of 200 logs. The test ran for 20 minutes, measuring the latency and throughput. |
There was a problem hiding this comment.
Logstash 7.13.2 was released in June '21. There have been +20 versions since then. Logstash is on 8.1.2 as of a few weeks ago. Can we run these on a more recent version? That will be a better comparison for the community.
There was a problem hiding this comment.
There have been some issues getting the latest versions of Logstash running in our performance testing environment. I propose that we revisit this on the next round of performance testing
performance-test/results/results.md
Outdated
|
|
||
|  | ||
|
|
||
| When comparing these measurements side by side, the test shows that with this configuration, Data Prepper has a `56%` higher throughput than Logstash, and an average latency that is `56%` lower than Logstash. |
There was a problem hiding this comment.
This uses Data Prepper as the subject and should be in the section above. Or we can add a separate conclusion section as well.
### Conclusion
Data Prepper has a `56%` higher throughput than Logstash, and an average latency that is `56%` lower than Logstash.
There was a problem hiding this comment.
The first image in this blog post is a very useful graph. Can we incorporate it?
https://opensearch.org/blog/technical-post/2022/02/data-prepper-1.2.1-performance-testing/
Also, it might be nice to include that graph or another very good summary at the very top. You have to go all the way to the bottom to get the results. I'd rather start with a good summary, then go into the details.
performance-test/results/results.md
Outdated
| @@ -0,0 +1,291 @@ | |||
| # Performance Test Results | |||
There was a problem hiding this comment.
I think we should move this into the /docs directory.
performance-test/results/results.md
Outdated
| @@ -0,0 +1,291 @@ | |||
| # Performance Test Results | |||
There was a problem hiding this comment.
Perhaps name this "Latest Performance Test Results" or "Current Performance Test Results."
performance-test/results/results.md
Outdated
| @@ -0,0 +1,291 @@ | |||
| # Performance Test Results | |||
|
|
|||
| Following the release of Data Prepper 1.3, performance tests were run for each of the new processors that were added, as well as the equivalent Logstash filters. In the performance test results discussed below, the test environments and configurations are identical, except where the same option is not available for all applications. | |||
There was a problem hiding this comment.
I'd like to make this page a living document which we update regularly (perhaps after certain releases). I think it would be good to reword the opening paragraph to clarify this.
This page tracks our latest performance test runs from Data Prepper. We run tests with significant processors, as well as the equivalent Logstash filters. In the performance test results discussed below, the test environments and configurations are identical, except where the same option is not available for all applications.
performance-test/results/results.md
Outdated
|
|
||
|  | ||
|
|
||
| Comparing the performance of the latest release of Data Prepper 1.3 against Logstash 7.13.2, the test was configured to simulate 10 clients to send requests as frequently as possible. Each request was contained a batch of 200 logs. The test ran for 20 minutes, measuring the latency and throughput. |
There was a problem hiding this comment.
Keeping in line with making this a living document, I think it would be better to not include version numbers in the paragraphs.
Instead, perhaps we have a table to show the versions used in the latest run.
performance-test/results/results.md
Outdated
|
|
||
| Following the release of Data Prepper 1.3, performance tests were run for each of the new processors that were added, as well as the equivalent Logstash filters. In the performance test results discussed below, the test environments and configurations are identical, except where the same option is not available for all applications. | ||
|
|
||
| ## Environment Details |
There was a problem hiding this comment.
Should we also clarify here or above that the test results are only for log ingestion? It is implied, but it might be good to make it explicit.
Signed-off-by: Taylor Gray <[email protected]>
I looked into creating a graph similar to the one in the performance testing blog post, and there is some extra effort to pull the data and format it the right way. For now, I moved the results to the start of the doc. |
Codecov Report
@@ Coverage Diff @@
## main #1309 +/- ##
=========================================
Coverage 94.08% 94.08%
Complexity 1156 1156
=========================================
Files 162 162
Lines 3248 3248
Branches 263 263
=========================================
Hits 3056 3056
Misses 138 138
Partials 54 54 Continue to review full report at Codecov.
|
Signed-off-by: Taylor Gray [email protected]
Description
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.