Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core/tracker: refactor inclusion checker #2120

Merged
merged 10 commits into from
Apr 24, 2023
Merged

Conversation

corverroos
Copy link
Contributor

@corverroos corverroos commented Apr 21, 2023

Refactors the inclusion delay tracker to also detect missed inclusions and to log broadcast delay of included and missed duties. The aim is to find a correlation between broadcast performance and inclusion delay.

category: feature
ticket: #1538

@codecov
Copy link

codecov bot commented Apr 21, 2023

Codecov Report

Patch coverage: 31.33% and project coverage change: -0.17 ⚠️

Comparison is base (139e45d) 53.52% compared to head (e1c841f) 53.36%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2120      +/-   ##
==========================================
- Coverage   53.52%   53.36%   -0.17%     
==========================================
  Files         173      173              
  Lines       22457    22618     +161     
==========================================
+ Hits        12021    12069      +48     
- Misses       8981     9095     +114     
+ Partials     1455     1454       -1     
Impacted Files Coverage Δ
app/app.go 7.64% <0.00%> (-0.05%) ⬇️
app/monitoringapi.go 51.16% <ø> (-0.29%) ⬇️
app/privkeylock/privkeylock.go 36.58% <ø> (-1.51%) ⬇️
cluster/definition.go 66.93% <ø> (ø)
core/tracking.go 0.00% <0.00%> (ø)
testutil/verifypr/verify.go 63.63% <ø> (ø)
core/tracker/inclusion.go 26.94% <26.94%> (ø)
app/eth2wrap/synthproposer.go 65.74% <80.76%> (-0.10%) ⬇️

... and 9 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

app/app.go Outdated Show resolved Hide resolved
core/tracker/incldelay.go Outdated Show resolved Hide resolved
@corverroos corverroos changed the title core/tracker: log broadcast vs inclusion delay core/tracker: refactor inclusion checker Apr 23, 2023
core/tracker/inclusion.go Outdated Show resolved Hide resolved
}
}

func reportAttInclusion(ctx context.Context, sub submission, block block) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need to pass full block, just pass blockSlot. AttSlot can be taken from submission.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also add godoc

Copy link
Contributor Author

@corverroos corverroos Apr 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aggregated logging field requires more info

}

// CheckBlock checks whether the block includes any of the submitted duties.
func (i *inclusion) CheckBlock(ctx context.Context, block block) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this is a public method?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is its API

}

// InclusionChecker checks whether duties have been included in blocks.
type InclusionChecker struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why you needed to wrap inclusion with InclusionChecker? Also some methods on inclusion struct are public which can be private as they only called by InclusionChecker methods which are exposed to other packages.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

renamed it to inclusionChecker which wrap a inclusionCore. The core is decoupled and optimised for easy testing.

continue // This duty is for another committee
}

if !att.AggregationBits.BitAt(duty.ValidatorCommitteeIndex) {
Copy link
Contributor

@dB2510 dB2510 Apr 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@corverroos you need to add this check in new inclusion checker. To check if our unaggregated attestation is included as part of an aggregated attestation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you haven't pushed that commit. It might be in local commit?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

z.I64("inclusion_delay", inclDelay),
z.Any("broadcast_delay", sub.Delay),
z.Int("aggregate_len", len(aggIndices)),
z.Bool("aggregated", len(aggIndices) > 1),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dB2510 the aggregated logging field is added here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to clarify, this aggregated logging will tell us that attestation is an aggregate attestation and NOT an unaggregated one. Well here aggregated: true only when we are aggregator of that slot AND it is our aggregate attestation.

Comment on lines +157 to +158
// TODO(corver): We should probably check if validator included in AggregationBits
// for attester duty by caching scheduler duty data to get validator committee index.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool, i was looking for this. Will push in next PR probably?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah

continue
}

// TODO(corver): We should probably check if validator included in AggregationBits
Copy link
Contributor

@xenowits xenowits Apr 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. If we wish, we can also understand if the attestation was aggregated or not by checking

if NO_OF_SET_BITS(attestation.aggregation_bits) == 1 { 
  "attestation was NOT aggregated by any aggregator" 
}

@corverroos corverroos added the merge when ready Indicates bulldozer bot may merge when all checks pass label Apr 24, 2023
@obol-bulldozer obol-bulldozer bot merged commit 976b4bc into main Apr 24, 2023
@obol-bulldozer obol-bulldozer bot deleted the corver/incldelta branch April 24, 2023 07:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merge when ready Indicates bulldozer bot may merge when all checks pass
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants