Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[cli] Improve span merging in the coverage tool #15120

Merged

Conversation

Rqnsom
Copy link
Contributor

@Rqnsom Rqnsom commented Oct 30, 2024

Description

  • Improve merging spans in the coverage tool
    • The old solution failed to merge neighbouring spans: e.g. [3, 5] and [5, 7] into [3, 7]
  • Also, make the merge_spans function public so it could be used by the move-mutation-test tool that also parses the coverage output file.

How Has This Been Tested?

  • A new unit test has been written.
  • cargo clippy hasn't introduced any new warnings for the new change.
  • I ran the aptos move coverage tool locally to ensure it still works as expected.

Key Areas to Review

This should have no visible impact on current aptos move coverage tool. The old limitation was not an issue, since displaying covered lines with always worked fine although spans were not merged "perfectly". This improvement is much more important for the usage in the move-mutation-test tool.

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Move Compiler
  • Other (specify)

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

Copy link

trunk-io bot commented Oct 30, 2024

⏱️ 15m total CI duration on this PR
Job Cumulative Duration Recent Runs
rust-move-tests 9m 🟩
check-dynamic-deps 2m 🟩
rust-cargo-deny 2m 🟩
general-lints 27s 🟩
permission-check 23s 🟥🟥🟩🟩🟥 (+2 more)
permission-check 21s 🟥🟥🟩🟩🟥 (+2 more)
semgrep/ci 20s 🟩
file_change_determinator 10s 🟩

settingsfeedbackdocs ⋅ learn more about trunk.io

@vineethk vineethk self-requested a review November 5, 2024 16:25
Copy link
Contributor

@vineethk vineethk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor comments, will approve once fixed.

@Rqnsom Rqnsom force-pushed the chore/improve-merge-spans-for-coverage branch from f0dc645 to edc960b Compare November 5, 2024 17:10
@Rqnsom Rqnsom requested a review from vineethk November 5, 2024 17:21
Copy link
Contributor

@vineethk vineethk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the comments!

Copy link
Contributor

@brmataptos brmataptos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks ok, though the coverage problems are deeper than this.

- Improve merging spans in the coverage tool:
  - The old solution failed to merge neighbouring spans
    e.g. `[3, 5]` and `[5, 7]` into `[3, 7]`
- Also, make the `merge_spans()` function public so it could be used by the
  `move-mutation-test` tool which also parses the coverage output file.
@Rqnsom Rqnsom force-pushed the chore/improve-merge-spans-for-coverage branch from 7ae1151 to 41c7a06 Compare November 6, 2024 07:48
@vineethk vineethk added the CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR label Nov 6, 2024

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

@vineethk vineethk enabled auto-merge (squash) November 14, 2024 20:51

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

✅ Forge suite realistic_env_max_load success on d7abff86c308f32ecbdbd7a23b94d4f831e557fd

two traffics test: inner traffic : committed: 14298.10 txn/s, latency: 2780.93 ms, (p50: 2700 ms, p70: 2700, p90: 3000 ms, p99: 4500 ms), latency samples: 5436500
two traffics test : committed: 100.08 txn/s, latency: 1457.15 ms, (p50: 1400 ms, p70: 1500, p90: 1500 ms, p99: 2400 ms), latency samples: 1720
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 2.048, avg: 1.576", "ConsensusProposalToOrdered: max: 0.319, avg: 0.293", "ConsensusOrderedToCommit: max: 0.387, avg: 0.374", "ConsensusProposalToCommit: max: 0.679, avg: 0.667"]
Max non-epoch-change gap was: 1 rounds at version 5009803 (avg 0.00) [limit 4], 1.80s no progress at version 5009803 (avg 0.20s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 8.59s no progress at version 1908718 (avg 8.30s) [limit 15].
Test Ok

Copy link
Contributor

✅ Forge suite framework_upgrade success on 2bb2d43037a93d883729869d65c7c6c75b028fa1 ==> d7abff86c308f32ecbdbd7a23b94d4f831e557fd

Compatibility test results for 2bb2d43037a93d883729869d65c7c6c75b028fa1 ==> d7abff86c308f32ecbdbd7a23b94d4f831e557fd (PR)
Upgrade the nodes to version: d7abff86c308f32ecbdbd7a23b94d4f831e557fd
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1344.46 txn/s, submitted: 1347.59 txn/s, failed submission: 3.13 txn/s, expired: 3.13 txn/s, latency: 2266.30 ms, (p50: 2100 ms, p70: 2400, p90: 3600 ms, p99: 5200 ms), latency samples: 120140
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1345.36 txn/s, submitted: 1347.99 txn/s, failed submission: 2.63 txn/s, expired: 2.63 txn/s, latency: 2184.70 ms, (p50: 1800 ms, p70: 2300, p90: 3600 ms, p99: 5800 ms), latency samples: 122840
5. check swarm health
Compatibility test for 2bb2d43037a93d883729869d65c7c6c75b028fa1 ==> d7abff86c308f32ecbdbd7a23b94d4f831e557fd passed
Upgrade the remaining nodes to version: d7abff86c308f32ecbdbd7a23b94d4f831e557fd
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1374.06 txn/s, submitted: 1377.00 txn/s, failed submission: 2.95 txn/s, expired: 2.95 txn/s, latency: 2445.43 ms, (p50: 2100 ms, p70: 2400, p90: 4100 ms, p99: 5700 ms), latency samples: 121200
Test Ok

Copy link
Contributor

✅ Forge suite compat success on 2bb2d43037a93d883729869d65c7c6c75b028fa1 ==> d7abff86c308f32ecbdbd7a23b94d4f831e557fd

Compatibility test results for 2bb2d43037a93d883729869d65c7c6c75b028fa1 ==> d7abff86c308f32ecbdbd7a23b94d4f831e557fd (PR)
1. Check liveness of validators at old version: 2bb2d43037a93d883729869d65c7c6c75b028fa1
compatibility::simple-validator-upgrade::liveness-check : committed: 14434.85 txn/s, latency: 2398.03 ms, (p50: 2000 ms, p70: 2100, p90: 3000 ms, p99: 8400 ms), latency samples: 542880
2. Upgrading first Validator to new version: d7abff86c308f32ecbdbd7a23b94d4f831e557fd
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 6850.84 txn/s, latency: 4119.22 ms, (p50: 4600 ms, p70: 4700, p90: 5000 ms, p99: 5600 ms), latency samples: 138200
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 7252.33 txn/s, latency: 4443.99 ms, (p50: 4600 ms, p70: 4800, p90: 6300 ms, p99: 6600 ms), latency samples: 242340
3. Upgrading rest of first batch to new version: d7abff86c308f32ecbdbd7a23b94d4f831e557fd
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 7652.22 txn/s, latency: 3752.58 ms, (p50: 4200 ms, p70: 4400, p90: 4400 ms, p99: 4600 ms), latency samples: 143200
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 7742.33 txn/s, latency: 4160.17 ms, (p50: 4400 ms, p70: 4500, p90: 5600 ms, p99: 6200 ms), latency samples: 257460
4. upgrading second batch to new version: d7abff86c308f32ecbdbd7a23b94d4f831e557fd
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 11775.65 txn/s, latency: 2325.65 ms, (p50: 2300 ms, p70: 2700, p90: 3200 ms, p99: 3400 ms), latency samples: 207800
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 10336.22 txn/s, latency: 2975.31 ms, (p50: 2300 ms, p70: 3100, p90: 6100 ms, p99: 8100 ms), latency samples: 336560
5. check swarm health
Compatibility test for 2bb2d43037a93d883729869d65c7c6c75b028fa1 ==> d7abff86c308f32ecbdbd7a23b94d4f831e557fd passed
Test Ok

@vineethk vineethk merged commit 5770aa4 into aptos-labs:main Nov 14, 2024
76 of 93 checks passed
@Rqnsom Rqnsom deleted the chore/improve-merge-spans-for-coverage branch November 15, 2024 07:20
Rqnsom added a commit to eigerco/move-spec-testing that referenced this pull request Nov 15, 2024
We recently merged a PR in aptos-core to improve their `merge_span`
function, and now we can use their implememntation:
aptos-labs/aptos-core#15120
Rqnsom added a commit to eigerco/move-spec-testing that referenced this pull request Nov 15, 2024
We recently merged a PR in aptos-core to improve their `merge_span`
function, and now we can use their implememntation:
aptos-labs/aptos-core#15120
Rqnsom added a commit to eigerco/move-spec-testing that referenced this pull request Nov 15, 2024
We recently merged a PR in aptos-core to improve their `merge_span`
function, and now we can use their implememntation:
aptos-labs/aptos-core#15120
Rqnsom added a commit to eigerco/move-spec-testing that referenced this pull request Nov 15, 2024
We recently merged a PR in aptos-core to improve their `merge_span`
function, and now we can use their implememntation:
aptos-labs/aptos-core#15120
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants