Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dev-docs: Add smoke tests docs to TESTING.md #8545

Merged
merged 3 commits into from
Jul 12, 2022

Conversation

marclop
Copy link
Contributor

@marclop marclop commented Jul 6, 2022

Motivation/summary

Adds a section about "smoke testing" in the TESTING.md.

Checklist

- [ ] Update CHANGELOG.asciidoc
- [ ] Update package changelog.yml (only if changes to apmpackage have been made)

  • Documentation has been updated

How to test these changes

N/A

Related issues

Part of #8303

Adds a section about "smoke testing" in the `TESTING.md`.

Signed-off-by: Marc Lopez Rubio <[email protected]>
@marclop marclop added docs backport-skip Skip notification from the automated backport with mergify v8.4.0 labels Jul 6, 2022
@marclop marclop requested a review from a team July 6, 2022 07:13
@apmmachine
Copy link
Contributor

apmmachine commented Jul 6, 2022

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-07-12T07:22:24.424+0000

  • Duration: 27 min 25 sec

Test stats 🧪

Test Results
Failed 0
Passed 4108
Skipped 13
Total 4121

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /hey-apm : Run the hey-apm benchmark.

  • /package : Generate and publish the docker images.

  • /test windows : Build & tests on Windows.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@apmmachine
Copy link
Contributor

apmmachine commented Jul 6, 2022

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (42/42) 💚
Files 91.96% (183/199) 👍
Classes 93.435% (427/457) 👍
Methods 89.163% (1086/1218) 👍 0.082
Lines 76.782% (13314/17340) 👎 -0.029
Conditionals 100.0% (0/0) 💚

@marclop marclop enabled auto-merge (squash) July 8, 2022 00:48
@mergify
Copy link
Contributor

mergify bot commented Jul 12, 2022

This pull request is now in conflicts. Could you fix it @marclop? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b f/add-smoke-testing-brief-documentation upstream/f/add-smoke-testing-brief-documentation
git merge upstream/main
git push upstream f/add-smoke-testing-brief-documentation

@apmmachine
Copy link
Contributor

📚 Go benchmark report

Diff with the main branch

name                                                                                            old time/op    new time/op    delta
pkg:github.com/elastic/apm-server/agentcfg goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/decoder goos:linux goarch:amd64
CompressedRequestReader/deflate_content_encoding-12                                               16.8µs ± 1%    17.0µs ± 1%   +1.44%  (p=0.016 n=5+4)
pkg:github.com/elastic/apm-server/model/modelindexer goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/processor/stream goos:linux goarch:amd64
BackendProcessor/minimal.ndjson-12                                                                63.7µs ±14%    46.4µs ±38%  -27.10%  (p=0.016 n=5+5)
BackendProcessor/otel-bridge.ndjson-12                                                            34.1µs ±13%    29.1µs ±15%  -14.65%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel0/invalid-event-type.ndjson-12           790ns ± 1%     810ns ± 2%   +2.52%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel0/unknown-span-type.ndjson-12           6.54µs ± 1%    6.63µs ± 1%   +1.32%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/ratelimit.ndjson-12                   59.3µs ±12%    50.2µs ±11%  -15.43%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/transactions-huge_traces.ndjson-12    20.3µs ±17%    16.3µs ± 3%  -19.67%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/transactions.ndjson-12                56.1µs ±23%    44.7µs ±19%  -20.36%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel4/invalid-json-event.ndjson-12          3.08µs ±11%    3.40µs ± 7%  +10.27%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/invalid-json-event.ndjson-12          1.66µs ± 0%    1.68µs ± 1%   +1.78%  (p=0.016 n=4+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/span-links.ndjson-12                  2.46µs ± 3%    2.71µs ± 2%  +10.15%  (p=0.008 n=5+5)
pkg:github.com/elastic/apm-server/publish goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/aggregation/spanmetrics goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/aggregation/txmetrics goos:linux goarch:amd64
AggregateTransaction-12                                                                           72.4ns ± 1%    70.9ns ± 1%   -2.10%  (p=0.008 n=5+5)
pkg:github.com/elastic/apm-server/x-pack/apm-server/sampling goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/sampling/eventstorage goos:linux goarch:amd64
ReadEvents/json_codec/100_events-12                                                                664µs ± 0%     675µs ± 3%   +1.70%  (p=0.032 n=5+5)

name                                                                                            old alloc/op   new alloc/op   delta
pkg:github.com/elastic/apm-server/agentcfg goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/decoder goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/model/modelindexer goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/processor/stream goos:linux goarch:amd64
BackendProcessor/invalid-metadata.ndjson-12                                                       2.94kB ± 2%    2.87kB ± 2%   -2.24%  (p=0.040 n=5+5)
BackendProcessor/transactions.ndjson-12                                                           59.3kB ± 1%    59.9kB ± 1%   +0.96%  (p=0.032 n=5+5)
BackendProcessor/unknown-span-type.ndjson-12                                                      26.4kB ± 1%    26.8kB ± 1%   +1.71%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel0/invalid-event.ndjson-12               10.3kB ± 1%    10.4kB ± 1%   +0.95%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel0/invalid-metadata-2.ndjson-12          3.58kB ± 2%    3.54kB ± 1%   -1.30%  (p=0.040 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/invalid-json-metadata.ndjson-12       5.83kB ± 3%    5.71kB ± 2%   -2.19%  (p=0.040 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/minimal-service.ndjson-12             12.0kB ± 0%    11.8kB ± 1%   -1.35%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/transactions_spans_rum_2.ndjson-12    12.9kB ± 1%    12.8kB ± 1%   -0.99%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel4/errors_2.ndjson-12                    39.3kB ± 1%    39.7kB ± 1%   +0.89%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/transactions_spans.ndjson-12          57.6kB ± 0%    58.0kB ± 1%   +0.61%  (p=0.032 n=5+5)
pkg:github.com/elastic/apm-server/publish goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/aggregation/spanmetrics goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/aggregation/txmetrics goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/sampling goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/sampling/eventstorage goos:linux goarch:amd64

name                                                                                            old allocs/op  new allocs/op  delta
pkg:github.com/elastic/apm-server/agentcfg goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/decoder goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/model/modelindexer goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/processor/stream goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/publish goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/aggregation/spanmetrics goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/aggregation/txmetrics goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/sampling goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/sampling/eventstorage goos:linux goarch:amd64

name                                                                                            old speed      new speed      delta
pkg:github.com/elastic/apm-server/processor/stream goos:linux goarch:amd64
BackendProcessor/minimal.ndjson-12                                                              16.4MB/s ±15%  23.6MB/s ±53%  +43.89%  (p=0.016 n=5+5)
BackendProcessor/otel-bridge.ndjson-12                                                          55.6MB/s ±12%  65.2MB/s ±14%  +17.28%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel0/invalid-event-type.ndjson-12         495MB/s ± 1%   482MB/s ± 2%   -2.46%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel0/unknown-span-type.ndjson-12          505MB/s ± 1%   499MB/s ± 1%   -1.30%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/ratelimit.ndjson-12                 71.6MB/s ±13%  84.2MB/s ±10%  +17.64%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/transactions-huge_traces.ndjson-12   158MB/s ±18%   195MB/s ± 3%  +22.96%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/transactions.ndjson-12               102MB/s ±21%   128MB/s ±17%  +24.53%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel4/invalid-json-event.ndjson-12         192MB/s ±12%   173MB/s ± 6%   -9.60%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/invalid-json-event.ndjson-12         355MB/s ± 0%   349MB/s ± 1%   -1.74%  (p=0.016 n=4+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/span-links.ndjson-12                 277MB/s ± 3%   252MB/s ± 2%   -9.23%  (p=0.008 n=5+5)

report generated with https://pkg.go.dev/golang.org/x/perf/cmd/benchstat

@marclop marclop merged commit 9b98013 into elastic:main Jul 12, 2022
@marclop marclop deleted the f/add-smoke-testing-brief-documentation branch July 12, 2022 07:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-skip Skip notification from the automated backport with mergify docs v8.4.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants