Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Network] Add simple queuing metrics for message sends. #15268

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

JoshLind
Copy link
Contributor

@JoshLind JoshLind commented Nov 13, 2024

Description

This PR adds simple queuing metrics for message sends to the network stack. Specifically, we now track the time between when a message is sent by an application (after serialization), until when it is finally written to the network wire.

The next set of PRs will include: (i) receiver side metrics; and (ii) support for messages with embedded timestamps for tracking transport times.

Testing Plan

Existing test infrastructure and manual verification, e.g., I hacked the code to force messages to be streamed. You can see the new metrics in the Network Queue Latencies section of the dashboard.

Copy link

trunk-io bot commented Nov 13, 2024

⏱️ 3h 57m total CI duration on this PR
Slowest 15 Jobs Cumulative Duration Recent Runs
test-target-determinator 1h 🟩🟩🟩🟩 (+10 more)
rust-cargo-deny 27m 🟩🟩🟩🟩 (+11 more)
check-dynamic-deps 25m 🟩🟩🟩🟩🟩 (+11 more)
rust-move-tests 10m 🟩
rust-move-tests 10m 🟩
rust-move-tests 10m 🟩
rust-move-tests 10m 🟩
rust-move-tests 9m 🟩
rust-move-tests 9m 🟩
semgrep/ci 9m 🟩🟩🟩🟩🟩 (+11 more)
rust-move-tests 8m 🟥
general-lints 7m 🟩🟩🟩🟩🟩 (+11 more)
rust-move-tests 7m 🟥
rust-move-tests 7m 🟥
rust-move-tests 7m 🟥

settingsfeedbackdocs ⋅ learn more about trunk.io

@JoshLind JoshLind added the CICD:run-forge-e2e-perf Run the e2e perf forge only label Nov 13, 2024

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

@JoshLind JoshLind force-pushed the net_queue_lat branch 2 times, most recently from 3910c19 to 8522b5a Compare November 14, 2024 03:26

This comment has been minimized.

This comment has been minimized.

@JoshLind JoshLind force-pushed the net_queue_lat branch 2 times, most recently from 18ffd3d to 669a73e Compare November 14, 2024 18:11

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

@JoshLind JoshLind force-pushed the net_queue_lat branch 2 times, most recently from e1e55b2 to 8589444 Compare November 15, 2024 01:50

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

@JoshLind JoshLind force-pushed the net_queue_lat branch 2 times, most recently from 5adeeff to 9af9761 Compare November 15, 2024 14:18

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

✅ Forge suite realistic_env_max_load success on 8a39b440229c6446a147f0fc5c2943ec981a723a

two traffics test: inner traffic : committed: 14506.93 txn/s, latency: 2743.94 ms, (p50: 2700 ms, p70: 2700, p90: 3000 ms, p99: 3600 ms), latency samples: 5516000
two traffics test : committed: 100.02 txn/s, latency: 2342.01 ms, (p50: 2100 ms, p70: 2100, p90: 2300 ms, p99: 11200 ms), latency samples: 1700
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 1.262, avg: 0.769", "ConsensusProposalToOrdered: max: 0.329, avg: 0.292", "ConsensusOrderedToCommit: max: 0.355, avg: 0.343", "ConsensusProposalToCommit: max: 0.644, avg: 0.636"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 1.16s no progress at version 2115341 (avg 0.20s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 8.55s no progress at version 2115339 (avg 7.17s) [limit 15].
Test Ok

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CICD:run-forge-e2e-perf Run the e2e perf forge only
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant