Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add blob count label to DATA_COLUMN_SIDECAR_COMPUTATION metric #6340

Merged
merged 4 commits into from
Sep 5, 2024
Merged

Add blob count label to DATA_COLUMN_SIDECAR_COMPUTATION metric #6340

merged 4 commits into from
Sep 5, 2024

Conversation

jimmygchen
Copy link
Member

@jimmygchen jimmygchen commented Sep 2, 2024 •
edited
Loading

Issue Addressed

The DATA_COLUMN_SIDECAR_COMPUTATION metric currently lacks granularity, as it doesn't differentiate by the number of blobs computed.

Proposed Changes

This PR introduces a blob count label to the DATA_COLUMN_SIDECAR_COMPUTATION metric. Additionally, it moves the metric timer into the compute function, ensuring that only successful computations are recorded, thereby improving the accuracy of the metrics.

Additionally the bucket is updated, so it captures times above 1s up to 10s.

@jimmygchen
Add blob count label to DATA_COLUMN_SIDECAR_COMPUTATION metric, and…
3d8a2a7 
… move metrics into the compute function, recording only successful computation.
@jimmygchen jimmygchen added das Data Availability Sampling ready-for-review The code is ready for review labels Sep 2, 2024
@jimmygchen jimmygchen added ready-for-merge This PR is ready to merge. and removed ready-for-review The code is ready for review labels Sep 4, 2024
@jimmygchen
Copy link
Member Author

@mergify queue

@mergify Mergify
Copy link

mergify bot commented Sep 4, 2024 •
edited
Loading

queue

🛑 The pull request has been removed from the queue default

The merge conditions cannot be satisfied due to failing checks.

You can take a look at Queue: Embarked in merge queue check runs for more details.

In case of a failure due to a flaky test, you should first retrigger the CI.
Then, re-embark the pull request into the merge queue by posting the comment
@mergifyio refresh on the pull request.

mergify bot added a commit that referenced this pull request Sep 4, 2024
@mergify
Merge of #6340
1624491
This was referenced Sep 4, 2024
Closed
Merged
Merged
mergify bot added a commit that referenced this pull request Sep 4, 2024
@mergify
Merge of #6340
24e5c99
@mergify mergify bot mentioned this pull request Sep 4, 2024
Closed
5 tasks
@michaelsproul
Copy link
Member

@mergify queue

@mergify Mergify
Copy link

mergify bot commented Sep 5, 2024

queue

🛑 The pull request has been removed from the queue default

The merge conditions cannot be satisfied due to failing checks.

You can take a look at Queue: Embarked in merge queue check runs for more details.

In case of a failure due to a flaky test, you should first retrigger the CI.
Then, re-embark the pull request into the merge queue by posting the comment
@mergifyio refresh on the pull request.

@michaelsproul
Copy link
Member

@mergify queue

@mergify Mergify
Copy link

mergify bot commented Sep 5, 2024 •
edited
Loading

queue

✅ The pull request has been merged automatically

The pull request has been merged automatically at b50ce60

mergify bot added a commit that referenced this pull request Sep 5, 2024
@mergify
Merge of #6340
7e52f01
@mergify mergify bot mentioned this pull request Sep 5, 2024
Closed
11 tasks
@mergify mergify bot merged commit b50ce60 into sigp:unstable Sep 5, 2024
28 checks passed
ThreeHrSleep added a commit to ThreeHrSleep/lighthouse that referenced this pull request Sep 8, 2024
@ThreeHrSleep
add tracing to validator_client
065e4ec 
Add retropgf funding (sigp#6324)

Migrate from `ethereum-types` to `alloy-primitives` (sigp#6078)

* Remove use of ethers_core::RlpStream

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into remove_use_of_ethers_core

* Remove old code

* Simplify keccak call

* Remove unused package

* Merge branch 'unstable' of https://github.com/ethDreamer/lighthouse into remove_use_of_ethers_core

* Merge branch 'unstable' into remove_use_of_ethers_core

* Run clippy

* Merge branch 'remove_use_of_ethers_core' of https://github.com/dospore/lighthouse into remove_use_of_ethers_core

* Check all cargo fmt

* migrate to alloy primitives init

* fix deps

* integrate alloy-primitives

* resolve dep issues

* more changes based on dep changes

* add TODOs

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into remove_use_of_ethers_core

* Revert lock

* Add BeaconBlocksByRange v3

* continue migration

* Revert "Add BeaconBlocksByRange v3"

This reverts commit e3ce7fc.

* impl hash256 extended trait

* revert some uneeded diffs

* merge conflict resolved

* fix subnet id rshift calc

* rename to FixedBytesExtended

* debugging

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into migrate-to-alloy-primitives

* fix failed test

* fixing more tests

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into remove_use_of_ethers_core

* introduce a shim to convert between the two u256 types

* move alloy to wrokspace

* align alloy versions

* update

* update web3signer test certs

* refactor

* resolve failing tests

* linting

* fix graffiti string test

* fmt

* fix ef test

* resolve merge conflicts

* remove udep and revert cert

* cargo patch

* cyclic dep

* fix build error

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into migrate-to-alloy-primitives

* resolve conflicts, update deps

* merge unstable

* fmt

* fix deps

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into migrate-to-alloy-primitives

* resolve merge conflicts

* resolve conflicts, make necessary changes

* Remove patch

* fmt

* remove file

* merge conflicts

* sneaking in a smol change

* bump versions

* Merge remote-tracking branch 'origin/unstable' into migrate-to-alloy-primitives

* Updates for peerDAS

* Update ethereum_hashing to prevent dupe

* updated alloy-consensus, removed TODOs

* cargo update

* endianess fix

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into migrate-to-alloy-primitives

* fmt

* fix merge

* fix test

* fixed_bytes crate

* minor fixes

* convert u256 to i64

* panic free mixin to_low_u64_le

* from_str_radix

* computbe_subnet api and ensuring we use big-endian

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into migrate-to-alloy-primitives

* fix test

* Simplify subnet_id test

* Simplify some more tests

* Add tests to fixed_bytes crate

* Merge branch 'unstable' into migrate-to-alloy-primitives

update libp2p to version 0.54 (sigp#6249)

* update libp2p to version 0.54.0

* address review

* Merge branch 'unstable' of github.com:sigp/lighthouse into update-libp2p

* Merge branch 'update-libp2p' of github.com:sigp/lighthouse into update-libp2p

Drop block data from BlockError and BlobError (sigp#5735)

* Drop block data from BlockError and BlobError

* Debug release tests

* Fix release tests

* Revert unnecessary changes to lighthouse_metrics

Update manual checkpoint sync content in Lighthouse book (sigp#6338)

* Update manual checkpiont sync

* Update faq

* Minor revision

* Minor revision

Light Client Bug Fix (sigp#6299)

* Light Client Bug Fix

Ignore Rust 1.82 warnings about void patterns (sigp#6357)

* Ignore Rust 1.82 warnings about void patterns

Enable `large_stack_frames` lint (sigp#6343)

* Enable `large_stack_frames` lint

Add blob count label to `DATA_COLUMN_SIDECAR_COMPUTATION` metric (sigp#6340)

* Add blob count label to `DATA_COLUMN_SIDECAR_COMPUTATION` metric, and move metrics into the compute function, recording only successful computation.

* Move `discard_timer_on_break` usage to caller site.

* Merge branch 'unstable' into compute-data-column-metric

* Merge branch 'unstable' into compute-data-column-metric

Metadata request ordering (sigp#6336)

* Send metadata request ordering

* Merge branch 'unstable' into metadata-order

Clarify validator monitor block log (sigp#6342)

* Clarify validator monitor block log

* Merge branch 'unstable' into clarify-block-log

Check known parent on rpc blob process (sigp#5893)

* Check known parent on rpc blob process

* fix test

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into blob-unknown-parent

Remove beta tag from gossipsub 1.2 (sigp#6344)

* Remove the beta tag from gossipsub v1.2

* fix clippy

* Merge branch 'unstable' into remove-beta-tag

Fix lints for Rust 1.81 (sigp#6363)

* Fix lints for Rust 1.81

Use tikv-jemallocator instead of jemallocator (sigp#6354)

* Use tikv-jemallocator instead of jemallocator

* Merge branch 'unstable' into tikv-jemallocator

* Bump tikv-jemallocator and tikv-jemalloc-ctl

Improve `get_custody_columns` validation, caching and error handling (sigp#6308)

* Improve `get_custody_columns` validation, caching and error handling.

* Merge branch 'unstable' into get-custody-columns-error-handing

* Fix failing test and add more test.

* Fix failing test and add more test.

* Merge branch 'unstable' into get-custody-columns-error-handing

* Add unit test to make sure the default specs won't panic on the `compute_custody_requirement_subnets` function.

* Add condition when calling `compute_custody_subnets_from_metadata` and update logs.

* Validate `csc` when returning from enr. Remove `csc` computation on connection since we get them on metadata anyway.

* Add `peers_per_custody_subnet_count` to track peer csc and supernodes.

* Disconnect peers with invalid metadata and find other peers instead.

* Fix sampling tests.

* Merge branch 'unstable' into get-custody-columns-error-handing

* Merge branch 'unstable' into get-custody-columns-error-handing

Delete legacy payload reconstruction (sigp#6213)

* Delete legacy payload reconstruction

* Delete unneeded failing test

* Merge remote-tracking branch 'origin/unstable' into remove-more-ethers

* Merge remote-tracking branch 'origin/unstable' into remove-more-ethers

* Cleanups

simplify rpc codec logic (sigp#6304)

* simplify rpc codec logic

* Merge branch 'unstable' of github.com:sigp/lighthouse into simplify-rpc-codec

* Merge branch 'unstable' of github.com:sigp/lighthouse into simplify-rpc-codec

* Merge branch 'unstable' of github.com:sigp/lighthouse into simply-rpc-codec

* Merge branch 'unstable' into simplify-rpc-codec

* Merge branch 'unstable' into simplify-rpc-codec
ThreeHrSleep added a commit to ThreeHrSleep/lighthouse that referenced this pull request Sep 22, 2024
@ThreeHrSleep
add tracing to validator_client
e5c107e 
Add retropgf funding (sigp#6324)

Migrate from `ethereum-types` to `alloy-primitives` (sigp#6078)

* Remove use of ethers_core::RlpStream

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into remove_use_of_ethers_core

* Remove old code

* Simplify keccak call

* Remove unused package

* Merge branch 'unstable' of https://github.com/ethDreamer/lighthouse into remove_use_of_ethers_core

* Merge branch 'unstable' into remove_use_of_ethers_core

* Run clippy

* Merge branch 'remove_use_of_ethers_core' of https://github.com/dospore/lighthouse into remove_use_of_ethers_core

* Check all cargo fmt

* migrate to alloy primitives init

* fix deps

* integrate alloy-primitives

* resolve dep issues

* more changes based on dep changes

* add TODOs

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into remove_use_of_ethers_core

* Revert lock

* Add BeaconBlocksByRange v3

* continue migration

* Revert "Add BeaconBlocksByRange v3"

This reverts commit e3ce7fc.

* impl hash256 extended trait

* revert some uneeded diffs

* merge conflict resolved

* fix subnet id rshift calc

* rename to FixedBytesExtended

* debugging

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into migrate-to-alloy-primitives

* fix failed test

* fixing more tests

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into remove_use_of_ethers_core

* introduce a shim to convert between the two u256 types

* move alloy to wrokspace

* align alloy versions

* update

* update web3signer test certs

* refactor

* resolve failing tests

* linting

* fix graffiti string test

* fmt

* fix ef test

* resolve merge conflicts

* remove udep and revert cert

* cargo patch

* cyclic dep

* fix build error

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into migrate-to-alloy-primitives

* resolve conflicts, update deps

* merge unstable

* fmt

* fix deps

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into migrate-to-alloy-primitives

* resolve merge conflicts

* resolve conflicts, make necessary changes

* Remove patch

* fmt

* remove file

* merge conflicts

* sneaking in a smol change

* bump versions

* Merge remote-tracking branch 'origin/unstable' into migrate-to-alloy-primitives

* Updates for peerDAS

* Update ethereum_hashing to prevent dupe

* updated alloy-consensus, removed TODOs

* cargo update

* endianess fix

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into migrate-to-alloy-primitives

* fmt

* fix merge

* fix test

* fixed_bytes crate

* minor fixes

* convert u256 to i64

* panic free mixin to_low_u64_le

* from_str_radix

* computbe_subnet api and ensuring we use big-endian

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into migrate-to-alloy-primitives

* fix test

* Simplify subnet_id test

* Simplify some more tests

* Add tests to fixed_bytes crate

* Merge branch 'unstable' into migrate-to-alloy-primitives

update libp2p to version 0.54 (sigp#6249)

* update libp2p to version 0.54.0

* address review

* Merge branch 'unstable' of github.com:sigp/lighthouse into update-libp2p

* Merge branch 'update-libp2p' of github.com:sigp/lighthouse into update-libp2p

Drop block data from BlockError and BlobError (sigp#5735)

* Drop block data from BlockError and BlobError

* Debug release tests

* Fix release tests

* Revert unnecessary changes to lighthouse_metrics

Update manual checkpoint sync content in Lighthouse book (sigp#6338)

* Update manual checkpiont sync

* Update faq

* Minor revision

* Minor revision

Light Client Bug Fix (sigp#6299)

* Light Client Bug Fix

Ignore Rust 1.82 warnings about void patterns (sigp#6357)

* Ignore Rust 1.82 warnings about void patterns

Enable `large_stack_frames` lint (sigp#6343)

* Enable `large_stack_frames` lint

Add blob count label to `DATA_COLUMN_SIDECAR_COMPUTATION` metric (sigp#6340)

* Add blob count label to `DATA_COLUMN_SIDECAR_COMPUTATION` metric, and move metrics into the compute function, recording only successful computation.

* Move `discard_timer_on_break` usage to caller site.

* Merge branch 'unstable' into compute-data-column-metric

* Merge branch 'unstable' into compute-data-column-metric

Metadata request ordering (sigp#6336)

* Send metadata request ordering

* Merge branch 'unstable' into metadata-order

Clarify validator monitor block log (sigp#6342)

* Clarify validator monitor block log

* Merge branch 'unstable' into clarify-block-log

Check known parent on rpc blob process (sigp#5893)

* Check known parent on rpc blob process

* fix test

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into blob-unknown-parent

Remove beta tag from gossipsub 1.2 (sigp#6344)

* Remove the beta tag from gossipsub v1.2

* fix clippy

* Merge branch 'unstable' into remove-beta-tag

Fix lints for Rust 1.81 (sigp#6363)

* Fix lints for Rust 1.81

Use tikv-jemallocator instead of jemallocator (sigp#6354)

* Use tikv-jemallocator instead of jemallocator

* Merge branch 'unstable' into tikv-jemallocator

* Bump tikv-jemallocator and tikv-jemalloc-ctl

Improve `get_custody_columns` validation, caching and error handling (sigp#6308)

* Improve `get_custody_columns` validation, caching and error handling.

* Merge branch 'unstable' into get-custody-columns-error-handing

* Fix failing test and add more test.

* Fix failing test and add more test.

* Merge branch 'unstable' into get-custody-columns-error-handing

* Add unit test to make sure the default specs won't panic on the `compute_custody_requirement_subnets` function.

* Add condition when calling `compute_custody_subnets_from_metadata` and update logs.

* Validate `csc` when returning from enr. Remove `csc` computation on connection since we get them on metadata anyway.

* Add `peers_per_custody_subnet_count` to track peer csc and supernodes.

* Disconnect peers with invalid metadata and find other peers instead.

* Fix sampling tests.

* Merge branch 'unstable' into get-custody-columns-error-handing

* Merge branch 'unstable' into get-custody-columns-error-handing

Delete legacy payload reconstruction (sigp#6213)

* Delete legacy payload reconstruction

* Delete unneeded failing test

* Merge remote-tracking branch 'origin/unstable' into remove-more-ethers

* Merge remote-tracking branch 'origin/unstable' into remove-more-ethers

* Cleanups

simplify rpc codec logic (sigp#6304)

* simplify rpc codec logic

* Merge branch 'unstable' of github.com:sigp/lighthouse into simplify-rpc-codec

* Merge branch 'unstable' of github.com:sigp/lighthouse into simplify-rpc-codec

* Merge branch 'unstable' of github.com:sigp/lighthouse into simply-rpc-codec

* Merge branch 'unstable' into simplify-rpc-codec

* Merge branch 'unstable' into simplify-rpc-codec
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@michaelsproul michaelsproul michaelsproul approved these changes

@dapplion dapplion dapplion approved these changes

Assignees
No one assigned
Labels
das Data Availability Sampling ready-for-merge This PR is ready to merge.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jimmygchen @michaelsproul @dapplion