Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revamp the end-of-test summary #4089

Draft
wants to merge 14 commits into
base: master
Choose a base branch
from

Conversation

joanlopez
Copy link
Contributor

@joanlopez joanlopez commented Dec 4, 2024

What?

It revamps the existing end-of-test summary to construct something that's more usabled, friendlier and cleaner than the current one.

Why?

Because the current one has been criticized for long, it really has its own deficiencies and it's good idea to revamp it before we turn k6 into an adult (v1).

Checklist

Before review readiness

  • Add support for a compact summary mode (likely enabled by default, so perhaps a "extended" flag, to avoid memory allocation issues), that only "relays" metrics but not stores metrics for groups and scenarios.
  • Revisit sorting: is there anything else we can sort for this first iteration (note that some ideas have been moved to a second iteration, see below)
  • Take a decision on what do we want to do with the existing summary, whether we want to use the new lib.Report as the new API for custom handleSummary (note this would be a breaking change), or if we want to ship this progressively.
    • Review and modify (if needed) the code in js/summary.go according to that decision.
  • Review and refactor the code in js/summary.js:
    • All the pieces from the old summary can probably be removed, if not removed yet.
    • De-duplicate the functions summarizeMetrics and summarizeMetricsWithThresholds, or just replace the first with the second one (as we may no longer need the first one if we remove the old summary code).
  • Review the structure and code of the output/summary package:
    • Although we kinda agreed this seems a good strategy, it would be nice to revisit how all this is defined and if there's any alternative way to achieve it (perhaps double check that with @.mstoykov).
  • Verify all scenarios (different test scripts) work as expected (test with no groups nor scenarios, test with only groups, test with only scenarios, test with the combination of both, all of them with compact mode enabled or not).
  • Make the CI look 🟢 as an 🍏 again!
  • (Optional) Define the JSDoc for the newly introduced functions in JS code.
  • (Optional) Re-write the JS code in TS.
  • Remove the playground/full-summary files, or define a proper location for them.
    --- Ideas left for a second iteration---
  • Sorting the thresholds by metric name, sorting the tags within the metric name and perhaps sorting the sources.
  • Implementing a more optimized data structure (as explored in https://github.com/joanlopez/xk6-custosummary/tree/main/timeseries) to use less memory allocations while keeping scenarios & groups samples.

General

  • I have performed a self-review of my code.
  • I have added tests for my changes.
  • I have run linter locally (make lint) and all checks pass.
  • I have run tests locally (make tests) and all tests pass.
  • I have commented on my code, particularly in hard-to-understand areas.

Related PR(s)/Issue(s)

@joanlopez joanlopez requested a review from a team as a code owner December 4, 2024 17:58
@joanlopez joanlopez requested review from mstoykov and olegbespalov and removed request for a team December 4, 2024 17:58
@joanlopez joanlopez marked this pull request as draft December 4, 2024 17:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants