Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Target Determination] Track JSON and YAML files. #15237

Merged
merged 1 commit into from
Nov 9, 2024
Merged

Conversation

JoshLind
Copy link
Contributor

@JoshLind JoshLind commented Nov 8, 2024

Description

This PR updates the target determinator to take into account JSON and YAML files. If any of these files change, the closest rust package is marked as affected. This should be good enough to catch any edge cases we've encountered (e.g., around the indexer GRPC tests).

Test Plan

Existing test infrastructure and manual verification. I changed a JSON file locally and verified that the closest package was marked as affected, e.g.,:

     Running `target/debug/aptos-cargo-cli affected-packages -vvv`
[2024-11-08T00:34:48Z INFO  aptos_cargo_cli::common] Identified the merge base: "7df77b85d4894c7355cb306abeecca5a4ded2d44"
[2024-11-08T00:34:48Z DEBUG aptos_cargo_cli::common] Attempting to read metadata from local file: ".../aptos-core/target/aptos-x-tool/metadata-7df77b85d4894c7355cb306abeecca5a4ded2d44.json"

...
Changed files detected:
        "ecosystem/indexer-grpc/indexer-test-transactions/json_transactions/imported_mainnet_txns/2739163_ans_current_ans_lookup.json"

...
Affected packages detected (1 total):
        "file:///.../aptos-core/ecosystem/indexer-grpc/indexer-test-transactions#aptos-indexer-test-transactions"

Copy link

trunk-io bot commented Nov 8, 2024

⏱️ 33m total CI duration on this PR
Slowest 15 Jobs Cumulative Duration Recent Runs
test-target-determinator 5m 🟩
rust-doc-tests 5m 🟩
execution-performance / test-target-determinator 5m 🟩
check-dynamic-deps 4m 🟩🟩
check 4m 🟩
rust-cargo-deny 4m 🟩🟩
rust-move-tests 2m 🟩
rust-move-tests 2m 🟩
fetch-last-released-docker-image-tag 2m 🟩
general-lints 52s 🟩🟩
semgrep/ci 45s 🟩🟩
file_change_determinator 24s 🟩🟩
file_change_determinator 11s 🟩
execution-performance / single-node-performance 10s 🟩
permission-check 6s 🟩🟩

🚨 2 jobs on the last run were significantly faster/slower than expected

Job Duration vs 7d avg Delta
check-dynamic-deps 2m 1m +102%
execution-performance / single-node-performance 10s 19m -99%

settingsfeedbackdocs ⋅ learn more about trunk.io

@JoshLind JoshLind requested review from geekflyer and aluon November 8, 2024 00:44
@JoshLind JoshLind enabled auto-merge (rebase) November 9, 2024 01:22

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

github-actions bot commented Nov 9, 2024

✅ Forge suite realistic_env_max_load success on db740c887cc59f14ae9c25ea673491a11131ebc1

two traffics test: inner traffic : committed: 14609.43 txn/s, latency: 2723.50 ms, (p50: 2700 ms, p70: 2700, p90: 2800 ms, p99: 3000 ms), latency samples: 5554840
two traffics test : committed: 99.99 txn/s, latency: 1692.90 ms, (p50: 1300 ms, p70: 1400, p90: 1600 ms, p99: 11200 ms), latency samples: 1760
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 1.998, avg: 1.557", "ConsensusProposalToOrdered: max: 0.312, avg: 0.290", "ConsensusOrderedToCommit: max: 0.354, avg: 0.345", "ConsensusProposalToCommit: max: 0.644, avg: 0.635"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.91s no progress at version 2949905 (avg 0.20s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 8.83s no progress at version 2949903 (avg 8.83s) [limit 15].
Test Ok

Copy link
Contributor

github-actions bot commented Nov 9, 2024

✅ Forge suite compat success on fcd2dedf6ca61a23f4979e944c629dcdbdae5dca ==> db740c887cc59f14ae9c25ea673491a11131ebc1

Compatibility test results for fcd2dedf6ca61a23f4979e944c629dcdbdae5dca ==> db740c887cc59f14ae9c25ea673491a11131ebc1 (PR)
1. Check liveness of validators at old version: fcd2dedf6ca61a23f4979e944c629dcdbdae5dca
compatibility::simple-validator-upgrade::liveness-check : committed: 15397.75 txn/s, latency: 2264.15 ms, (p50: 1900 ms, p70: 2100, p90: 2200 ms, p99: 7500 ms), latency samples: 564580
2. Upgrading first Validator to new version: db740c887cc59f14ae9c25ea673491a11131ebc1
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 8271.92 txn/s, latency: 3469.05 ms, (p50: 3900 ms, p70: 4000, p90: 4200 ms, p99: 4200 ms), latency samples: 150180
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 7448.39 txn/s, latency: 4221.66 ms, (p50: 4100 ms, p70: 4200, p90: 6800 ms, p99: 7200 ms), latency samples: 248980
3. Upgrading rest of first batch to new version: db740c887cc59f14ae9c25ea673491a11131ebc1
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 8180.38 txn/s, latency: 3481.63 ms, (p50: 3900 ms, p70: 4100, p90: 4200 ms, p99: 4300 ms), latency samples: 149660
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 7481.79 txn/s, latency: 3933.39 ms, (p50: 4100 ms, p70: 4200, p90: 4300 ms, p99: 5300 ms), latency samples: 277440
4. upgrading second batch to new version: db740c887cc59f14ae9c25ea673491a11131ebc1
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 12651.58 txn/s, latency: 2104.94 ms, (p50: 2200 ms, p70: 2400, p90: 2600 ms, p99: 2700 ms), latency samples: 220100
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 11873.90 txn/s, latency: 2603.44 ms, (p50: 2300 ms, p70: 2600, p90: 4400 ms, p99: 5800 ms), latency samples: 381300
5. check swarm health
Compatibility test for fcd2dedf6ca61a23f4979e944c629dcdbdae5dca ==> db740c887cc59f14ae9c25ea673491a11131ebc1 passed
Test Ok

@JoshLind JoshLind merged commit dff834c into main Nov 9, 2024
94 checks passed
@JoshLind JoshLind deleted the tar_det_indexer branch November 9, 2024 01:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants