Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: handle outOfRangeData when range sync Deneb #7249

Merged
merged 4 commits into from
Nov 27, 2024
Merged

fix: handle outOfRangeData when range sync Deneb #7249

merged 4 commits into from
Nov 27, 2024

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Nov 27, 2024

Motivation

Cannot sync lodestar from mekong genesis due to this error Cannot sync blobs outside of blobs prune window

Description

Beacon node should return BlockInputType.outOfRangeData block input instead of throwing that error

Closes #7248

@twoeths twoeths requested a review from a team as a code owner November 27, 2024 03:20
@twoeths twoeths changed the title fix: handle outOfRangeData for beaconBlocksMaybeBlobsByRange() fix: handle outOfRangeData when range sync Deneb Nov 27, 2024
nflaig
nflaig previously approved these changes Nov 27, 2024
View reviewed changes
@codecov Codecov
Copy link

codecov bot commented Nov 27, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 0% with 3 lines in your changes missing coverage. Please review.

Project coverage is 48.51%. Comparing base (18f4218) to head (a373b16).
Report is 3 commits behind head on unstable.

Additional details and impacted files 
@@             Coverage Diff              @@
##           unstable    #7249      +/-   ##
============================================
- Coverage     48.51%   48.51%   -0.01%     
============================================
  Files           600      600              
  Lines         40121    40122       +1     
  Branches       2054     2054              
============================================
  Hits          19466    19466              
- Misses        20617    20618       +1     
  Partials         38       38

@twoeths twoeths marked this pull request as draft November 27, 2024 07:58
@twoeths
Copy link
Contributor Author

twoeths commented Nov 27, 2024

not a complete fix, got

Nov-27 06:31:46.242[chain]           error: Error processing finalized checkpoint epoch=132619 - No block found for slot 4243744 root 0x33644e93138cf253cd809458ca2022213ecc282aab3f2f98faaab3de117cc1ec
Nov-27 06:31:55.124[chain]           error: Error processing finalized checkpoint epoch=132620 - No block found for slot 4243744 root 0x33644e93138cf253cd809458ca2022213ecc282aab3f2f98faaab3de117cc1ec

the reason is all archiveBlocks never finish, and the archiveBlocks for epoch n try to reprocess blocks of epoch n-1. The first failed is

Nov-27 03:13:26.363[chain]           error: Error processing finalized checkpoint epoch=2 - No blobSidecars found for slot 64 root 0xe845c6383893d2ac7e4ec87a3cb836ccaaa4ff9a73f45e64d04ff2f0bb3ddd00

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 27, 2024 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 895bc53 Previous: 18f4218 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.0592 ms/op 1.9524 ms/op 1.05
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 57.397 us/op 42.152 us/op 1.36
BLS verify - blst 861.08 us/op 881.18 us/op 0.98
BLS verifyMultipleSignatures 3 - blst 1.2873 ms/op 1.7449 ms/op 0.74
BLS verifyMultipleSignatures 8 - blst 1.7603 ms/op 2.0996 ms/op 0.84
BLS verifyMultipleSignatures 32 - blst 5.0661 ms/op 4.4837 ms/op 1.13
BLS verifyMultipleSignatures 64 - blst 9.4246 ms/op 9.0045 ms/op 1.05
BLS verifyMultipleSignatures 128 - blst 18.728 ms/op 16.094 ms/op 1.16
BLS deserializing 10000 signatures 738.93 ms/op 631.17 ms/op 1.17
BLS deserializing 100000 signatures 7.3420 s/op 6.3028 s/op 1.16
BLS verifyMultipleSignatures - same message - 3 - blst 973.26 us/op 1.0428 ms/op 0.93
BLS verifyMultipleSignatures - same message - 8 - blst 1.1012 ms/op 1.1274 ms/op 0.98
BLS verifyMultipleSignatures - same message - 32 - blst 1.8294 ms/op 1.6910 ms/op 1.08
BLS verifyMultipleSignatures - same message - 64 - blst 2.7589 ms/op 2.5148 ms/op 1.10
BLS verifyMultipleSignatures - same message - 128 - blst 4.5823 ms/op 4.1142 ms/op 1.11
BLS aggregatePubkeys 32 - blst 20.450 us/op 18.158 us/op 1.13
BLS aggregatePubkeys 128 - blst 73.437 us/op 64.005 us/op 1.15
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 62.622 ms/op 83.374 ms/op 0.75
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 62.255 ms/op 65.261 ms/op 0.95
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 45.885 ms/op 41.730 ms/op 1.10
getSlashingsAndExits - default max 120.83 us/op 74.184 us/op 1.63
getSlashingsAndExits - 2k 292.10 us/op 336.02 us/op 0.87
proposeBlockBody type=full, size=empty 6.5057 ms/op 6.1156 ms/op 1.06
isKnown best case - 1 super set check 351.00 ns/op 599.00 ns/op 0.59
isKnown normal case - 2 super set checks 329.00 ns/op 561.00 ns/op 0.59
isKnown worse case - 16 super set checks 363.00 ns/op 637.00 ns/op 0.57
InMemoryCheckpointStateCache - add get delete 3.2570 us/op 2.7810 us/op 1.17
validate api signedAggregateAndProof - struct 1.4669 ms/op 1.8835 ms/op 0.78
validate gossip signedAggregateAndProof - struct 1.4188 ms/op 1.8017 ms/op 0.79
batch validate gossip attestation - vc 640000 - chunk 32 133.26 us/op 130.01 us/op 1.02
batch validate gossip attestation - vc 640000 - chunk 64 115.52 us/op 113.79 us/op 1.02
batch validate gossip attestation - vc 640000 - chunk 128 111.29 us/op 115.57 us/op 0.96
batch validate gossip attestation - vc 640000 - chunk 256 109.37 us/op 106.34 us/op 1.03
pickEth1Vote - no votes 1.1338 ms/op 864.90 us/op 1.31
pickEth1Vote - max votes 6.9831 ms/op 5.7705 ms/op 1.21
pickEth1Vote - Eth1Data hashTreeRoot value x2048 15.708 ms/op 11.654 ms/op 1.35
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 18.594 ms/op 18.539 ms/op 1.00
pickEth1Vote - Eth1Data fastSerialize value x2048 530.66 us/op 370.63 us/op 1.43
pickEth1Vote - Eth1Data fastSerialize tree x2048 2.6589 ms/op 3.5914 ms/op 0.74
bytes32 toHexString 500.00 ns/op 632.00 ns/op 0.79
bytes32 Buffer.toString(hex) 276.00 ns/op 468.00 ns/op 0.59
bytes32 Buffer.toString(hex) from Uint8Array 430.00 ns/op 558.00 ns/op 0.77
bytes32 Buffer.toString(hex) + 0x 280.00 ns/op 480.00 ns/op 0.58
Object access 1 prop 0.16600 ns/op 0.33600 ns/op 0.49
Map access 1 prop 0.14700 ns/op 0.32300 ns/op 0.46
Object get x1000 6.2860 ns/op 5.6860 ns/op 1.11
Map get x1000 6.7050 ns/op 6.6140 ns/op 1.01
Object set x1000 36.583 ns/op 29.109 ns/op 1.26
Map set x1000 24.992 ns/op 20.276 ns/op 1.23
Return object 10000 times 0.30650 ns/op 0.31150 ns/op 0.98
Throw Error 10000 times 4.1509 us/op 2.9703 us/op 1.40
toHex 170.44 ns/op 141.10 ns/op 1.21
Buffer.from 157.44 ns/op 120.56 ns/op 1.31
shared Buffer 103.83 ns/op 79.098 ns/op 1.31
fastMsgIdFn sha256 / 200 bytes 2.3410 us/op 2.1850 us/op 1.07
fastMsgIdFn h32 xxhash / 200 bytes 281.00 ns/op 521.00 ns/op 0.54
fastMsgIdFn h64 xxhash / 200 bytes 289.00 ns/op 484.00 ns/op 0.60
fastMsgIdFn sha256 / 1000 bytes 7.6430 us/op 5.9110 us/op 1.29
fastMsgIdFn h32 xxhash / 1000 bytes 431.00 ns/op 651.00 ns/op 0.66
fastMsgIdFn h64 xxhash / 1000 bytes 384.00 ns/op 581.00 ns/op 0.66
fastMsgIdFn sha256 / 10000 bytes 72.679 us/op 50.992 us/op 1.43
fastMsgIdFn h32 xxhash / 10000 bytes 2.1050 us/op 2.1020 us/op 1.00
fastMsgIdFn h64 xxhash / 10000 bytes 1.3160 us/op 1.4220 us/op 0.93
send data - 1000 256B messages 13.977 ms/op 13.138 ms/op 1.06
send data - 1000 512B messages 20.138 ms/op 20.279 ms/op 0.99
send data - 1000 1024B messages 29.261 ms/op 30.035 ms/op 0.97
send data - 1000 1200B messages 28.083 ms/op 30.757 ms/op 0.91
send data - 1000 2048B messages 34.640 ms/op 31.605 ms/op 1.10
send data - 1000 4096B messages 32.258 ms/op 32.852 ms/op 0.98
send data - 1000 16384B messages 82.015 ms/op 71.452 ms/op 1.15
send data - 1000 65536B messages 226.67 ms/op 341.02 ms/op 0.66
enrSubnets - fastDeserialize 64 bits 1.1840 us/op 1.2060 us/op 0.98
enrSubnets - ssz BitVector 64 bits 392.00 ns/op 654.00 ns/op 0.60
enrSubnets - fastDeserialize 4 bits 187.00 ns/op 426.00 ns/op 0.44
enrSubnets - ssz BitVector 4 bits 410.00 ns/op 650.00 ns/op 0.63
prioritizePeers score -10:0 att 32-0.1 sync 2-0 162.51 us/op 183.11 us/op 0.89
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 159.67 us/op 148.52 us/op 1.08
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 317.89 us/op 270.86 us/op 1.17
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 413.63 us/op 424.28 us/op 0.97
prioritizePeers score 0:0 att 64-1 sync 4-1 675.97 us/op 721.93 us/op 0.94
array of 16000 items push then shift 1.7343 us/op 1.3118 us/op 1.32
LinkedList of 16000 items push then shift 7.8060 ns/op 8.3460 ns/op 0.94
array of 16000 items push then pop 124.05 ns/op 123.44 ns/op 1.00
LinkedList of 16000 items push then pop 7.5970 ns/op 6.9130 ns/op 1.10
array of 24000 items push then shift 2.4760 us/op 1.9320 us/op 1.28
LinkedList of 24000 items push then shift 7.7170 ns/op 6.6210 ns/op 1.17
array of 24000 items push then pop 153.58 ns/op 118.67 ns/op 1.29
LinkedList of 24000 items push then pop 7.7640 ns/op 6.3020 ns/op 1.23
intersect bitArray bitLen 8 6.7900 ns/op 5.7850 ns/op 1.17
intersect array and set length 8 50.242 ns/op 38.262 ns/op 1.31
intersect bitArray bitLen 128 31.160 ns/op 26.919 ns/op 1.16
intersect array and set length 128 751.59 ns/op 675.21 ns/op 1.11
bitArray.getTrueBitIndexes() bitLen 128 2.0390 us/op 2.3200 us/op 0.88
bitArray.getTrueBitIndexes() bitLen 248 3.5120 us/op 3.5150 us/op 1.00
bitArray.getTrueBitIndexes() bitLen 512 7.8610 us/op 7.1310 us/op 1.10
Buffer.concat 32 items 948.00 ns/op 1.0560 us/op 0.90
Uint8Array.set 32 items 1.8370 us/op 1.4470 us/op 1.27
Buffer.copy 1.7570 us/op 2.3660 us/op 0.74
Uint8Array.set - with subarray 2.6970 us/op 2.8830 us/op 0.94
Uint8Array.set - without subarray 1.3980 us/op 2.2970 us/op 0.61
getUint32 - dataview 261.00 ns/op 460.00 ns/op 0.57
getUint32 - manual 173.00 ns/op 354.00 ns/op 0.49
Set add up to 64 items then delete first 2.2745 us/op 1.7920 us/op 1.27
OrderedSet add up to 64 items then delete first 3.3864 us/op 2.8668 us/op 1.18
Set add up to 64 items then delete last 2.6431 us/op 2.1009 us/op 1.26
OrderedSet add up to 64 items then delete last 4.0757 us/op 3.0898 us/op 1.32
Set add up to 64 items then delete middle 2.7770 us/op 2.0410 us/op 1.36
OrderedSet add up to 64 items then delete middle 5.6200 us/op 4.5580 us/op 1.23
Set add up to 128 items then delete first 5.2605 us/op 4.0353 us/op 1.30
OrderedSet add up to 128 items then delete first 7.7095 us/op 6.3447 us/op 1.22
Set add up to 128 items then delete last 5.2640 us/op 3.9750 us/op 1.32
OrderedSet add up to 128 items then delete last 8.1087 us/op 5.9651 us/op 1.36
Set add up to 128 items then delete middle 5.1524 us/op 3.8608 us/op 1.33
OrderedSet add up to 128 items then delete middle 14.457 us/op 12.052 us/op 1.20
Set add up to 256 items then delete first 10.505 us/op 7.9101 us/op 1.33
OrderedSet add up to 256 items then delete first 17.015 us/op 12.341 us/op 1.38
Set add up to 256 items then delete last 11.121 us/op 7.6223 us/op 1.46
OrderedSet add up to 256 items then delete last 17.043 us/op 11.674 us/op 1.46
Set add up to 256 items then delete middle 10.511 us/op 7.5639 us/op 1.39
OrderedSet add up to 256 items then delete middle 42.247 us/op 34.724 us/op 1.22
transfer serialized Status (84 B) 1.5560 us/op 1.3930 us/op 1.12
copy serialized Status (84 B) 1.1440 us/op 1.2000 us/op 0.95
transfer serialized SignedVoluntaryExit (112 B) 1.6780 us/op 1.4940 us/op 1.12
copy serialized SignedVoluntaryExit (112 B) 1.2820 us/op 1.2670 us/op 1.01
transfer serialized ProposerSlashing (416 B) 1.9200 us/op 1.6320 us/op 1.18
copy serialized ProposerSlashing (416 B) 2.1130 us/op 1.8190 us/op 1.16
transfer serialized Attestation (485 B) 2.3400 us/op 1.7710 us/op 1.32
copy serialized Attestation (485 B) 2.4620 us/op 1.6600 us/op 1.48
transfer serialized AttesterSlashing (33232 B) 2.4800 us/op 1.6620 us/op 1.49
copy serialized AttesterSlashing (33232 B) 6.0430 us/op 4.4390 us/op 1.36
transfer serialized Small SignedBeaconBlock (128000 B) 2.9290 us/op 2.4300 us/op 1.21
copy serialized Small SignedBeaconBlock (128000 B) 17.146 us/op 11.083 us/op 1.55
transfer serialized Avg SignedBeaconBlock (200000 B) 3.6200 us/op 3.8480 us/op 0.94
copy serialized Avg SignedBeaconBlock (200000 B) 25.376 us/op 33.781 us/op 0.75
transfer serialized BlobsSidecar (524380 B) 3.7990 us/op 5.0600 us/op 0.75
copy serialized BlobsSidecar (524380 B) 81.878 us/op 96.778 us/op 0.85
transfer serialized Big SignedBeaconBlock (1000000 B) 3.6780 us/op 4.2620 us/op 0.86
copy serialized Big SignedBeaconBlock (1000000 B) 157.93 us/op 170.37 us/op 0.93
pass gossip attestations to forkchoice per slot 3.0585 ms/op 2.6393 ms/op 1.16
forkChoice updateHead vc 100000 bc 64 eq 0 537.80 us/op 362.03 us/op 1.49
forkChoice updateHead vc 600000 bc 64 eq 0 4.0881 ms/op 2.3142 ms/op 1.77
forkChoice updateHead vc 1000000 bc 64 eq 0 5.2733 ms/op 3.7612 ms/op 1.40
forkChoice updateHead vc 600000 bc 320 eq 0 3.0668 ms/op 2.2329 ms/op 1.37
forkChoice updateHead vc 600000 bc 1200 eq 0 3.1077 ms/op 2.2768 ms/op 1.36
forkChoice updateHead vc 600000 bc 7200 eq 0 3.8081 ms/op 3.2816 ms/op 1.16
forkChoice updateHead vc 600000 bc 64 eq 1000 11.043 ms/op 9.8071 ms/op 1.13
forkChoice updateHead vc 600000 bc 64 eq 10000 10.853 ms/op 10.598 ms/op 1.02
forkChoice updateHead vc 600000 bc 64 eq 300000 15.457 ms/op 33.287 ms/op 0.46
computeDeltas 500000 validators 300 proto nodes 4.2998 ms/op 3.9676 ms/op 1.08
computeDeltas 500000 validators 1200 proto nodes 4.3937 ms/op 4.4810 ms/op 0.98
computeDeltas 500000 validators 7200 proto nodes 4.5498 ms/op 4.4845 ms/op 1.01
computeDeltas 750000 validators 300 proto nodes 6.5501 ms/op 5.9240 ms/op 1.11
computeDeltas 750000 validators 1200 proto nodes 6.5314 ms/op 5.9333 ms/op 1.10
computeDeltas 750000 validators 7200 proto nodes 6.6304 ms/op 5.6229 ms/op 1.18
computeDeltas 1400000 validators 300 proto nodes 12.402 ms/op 11.291 ms/op 1.10
computeDeltas 1400000 validators 1200 proto nodes 12.979 ms/op 10.959 ms/op 1.18
computeDeltas 1400000 validators 7200 proto nodes 12.468 ms/op 11.094 ms/op 1.12
computeDeltas 2100000 validators 300 proto nodes 18.563 ms/op 15.617 ms/op 1.19
computeDeltas 2100000 validators 1200 proto nodes 18.024 ms/op 14.922 ms/op 1.21
computeDeltas 2100000 validators 7200 proto nodes 17.854 ms/op 14.947 ms/op 1.19
altair processAttestation - 250000 vs - 7PWei normalcase 1.9388 ms/op 1.8098 ms/op 1.07
altair processAttestation - 250000 vs - 7PWei worstcase 3.0004 ms/op 2.7148 ms/op 1.11
altair processAttestation - setStatus - 1/6 committees join 103.77 us/op 69.999 us/op 1.48
altair processAttestation - setStatus - 1/3 committees join 197.24 us/op 138.01 us/op 1.43
altair processAttestation - setStatus - 1/2 committees join 276.60 us/op 223.06 us/op 1.24
altair processAttestation - setStatus - 2/3 committees join 346.73 us/op 298.36 us/op 1.16
altair processAttestation - setStatus - 4/5 committees join 496.39 us/op 432.70 us/op 1.15
altair processAttestation - setStatus - 100% committees join 601.97 us/op 534.07 us/op 1.13
altair processBlock - 250000 vs - 7PWei normalcase 4.8062 ms/op 6.5070 ms/op 0.74
altair processBlock - 250000 vs - 7PWei normalcase hashState 26.641 ms/op 27.734 ms/op 0.96
altair processBlock - 250000 vs - 7PWei worstcase 36.160 ms/op 39.817 ms/op 0.91
altair processBlock - 250000 vs - 7PWei worstcase hashState 70.467 ms/op 74.339 ms/op 0.95
phase0 processBlock - 250000 vs - 7PWei normalcase 1.9387 ms/op 1.7987 ms/op 1.08
phase0 processBlock - 250000 vs - 7PWei worstcase 22.792 ms/op 24.167 ms/op 0.94
altair processEth1Data - 250000 vs - 7PWei normalcase 340.52 us/op 272.46 us/op 1.25
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 6.7530 us/op 6.8150 us/op 0.99
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 41.738 us/op 35.235 us/op 1.18
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 10.959 us/op 10.994 us/op 1.00
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 7.9410 us/op 7.4530 us/op 1.07
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 144.30 us/op 130.70 us/op 1.10
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.1488 ms/op 942.07 us/op 1.22
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.6539 ms/op 1.2373 ms/op 1.34
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.5317 ms/op 1.1865 ms/op 1.29
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 5.2022 ms/op 2.9855 ms/op 1.74
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.6411 ms/op 1.3108 ms/op 1.25
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 3.9593 ms/op 3.3390 ms/op 1.19
Tree 40 250000 create 231.52 ms/op 205.63 ms/op 1.13
Tree 40 250000 get(125000) 155.03 ns/op 120.17 ns/op 1.29
Tree 40 250000 set(125000) 738.71 ns/op 544.81 ns/op 1.36
Tree 40 250000 toArray() 17.448 ms/op 18.600 ms/op 0.94
Tree 40 250000 iterate all - toArray() + loop 17.868 ms/op 19.371 ms/op 0.92
Tree 40 250000 iterate all - get(i) 54.177 ms/op 56.823 ms/op 0.95
Array 250000 create 3.2612 ms/op 3.5482 ms/op 0.92
Array 250000 clone - spread 1.6045 ms/op 1.4740 ms/op 1.09
Array 250000 get(125000) 0.44000 ns/op 0.67400 ns/op 0.65
Array 250000 set(125000) 0.46500 ns/op 0.66100 ns/op 0.70
Array 250000 iterate all - loop 84.100 us/op 84.189 us/op 1.00
phase0 afterProcessEpoch - 250000 vs - 7PWei 50.625 ms/op 46.251 ms/op 1.09
Array.fill - length 1000000 3.8054 ms/op 2.7042 ms/op 1.41
Array push - length 1000000 13.810 ms/op 19.503 ms/op 0.71
Array.get 0.28413 ns/op 0.27241 ns/op 1.04
Uint8Array.get 0.45600 ns/op 0.34837 ns/op 1.31
phase0 beforeProcessEpoch - 250000 vs - 7PWei 20.887 ms/op 14.886 ms/op 1.40
altair processEpoch - mainnet_e81889 282.83 ms/op 239.57 ms/op 1.18
mainnet_e81889 - altair beforeProcessEpoch 20.152 ms/op 18.722 ms/op 1.08
mainnet_e81889 - altair processJustificationAndFinalization 21.035 us/op 16.217 us/op 1.30
mainnet_e81889 - altair processInactivityUpdates 5.9147 ms/op 5.7208 ms/op 1.03
mainnet_e81889 - altair processRewardsAndPenalties 49.967 ms/op 41.907 ms/op 1.19
mainnet_e81889 - altair processRegistryUpdates 3.0480 us/op 2.5940 us/op 1.18
mainnet_e81889 - altair processSlashings 362.00 ns/op 1.2780 us/op 0.28
mainnet_e81889 - altair processEth1DataReset 561.00 ns/op 902.00 ns/op 0.62
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.9799 ms/op 1.1205 ms/op 1.77
mainnet_e81889 - altair processSlashingsReset 3.5620 us/op 4.3230 us/op 0.82
mainnet_e81889 - altair processRandaoMixesReset 5.9000 us/op 6.9260 us/op 0.85
mainnet_e81889 - altair processHistoricalRootsUpdate 523.00 ns/op 1.2960 us/op 0.40
mainnet_e81889 - altair processParticipationFlagUpdates 3.1760 us/op 3.9950 us/op 0.79
mainnet_e81889 - altair processSyncCommitteeUpdates 433.00 ns/op 902.00 ns/op 0.48
mainnet_e81889 - altair afterProcessEpoch 53.636 ms/op 44.422 ms/op 1.21
capella processEpoch - mainnet_e217614 1.0701 s/op 1.0892 s/op 0.98
mainnet_e217614 - capella beforeProcessEpoch 79.745 ms/op 71.670 ms/op 1.11
mainnet_e217614 - capella processJustificationAndFinalization 17.344 us/op 14.652 us/op 1.18
mainnet_e217614 - capella processInactivityUpdates 19.089 ms/op 18.642 ms/op 1.02
mainnet_e217614 - capella processRewardsAndPenalties 213.78 ms/op 231.55 ms/op 0.92
mainnet_e217614 - capella processRegistryUpdates 14.899 us/op 15.166 us/op 0.98
mainnet_e217614 - capella processSlashings 632.00 ns/op 755.00 ns/op 0.84
mainnet_e217614 - capella processEth1DataReset 463.00 ns/op 969.00 ns/op 0.48
mainnet_e217614 - capella processEffectiveBalanceUpdates 15.505 ms/op 17.095 ms/op 0.91
mainnet_e217614 - capella processSlashingsReset 5.2250 us/op 4.8910 us/op 1.07
mainnet_e217614 - capella processRandaoMixesReset 8.7460 us/op 3.7280 us/op 2.35
mainnet_e217614 - capella processHistoricalRootsUpdate 717.00 ns/op 812.00 ns/op 0.88
mainnet_e217614 - capella processParticipationFlagUpdates 2.6980 us/op 1.9490 us/op 1.38
mainnet_e217614 - capella afterProcessEpoch 128.38 ms/op 106.48 ms/op 1.21
phase0 processEpoch - mainnet_e58758 351.94 ms/op 288.65 ms/op 1.22
mainnet_e58758 - phase0 beforeProcessEpoch 69.313 ms/op 89.901 ms/op 0.77
mainnet_e58758 - phase0 processJustificationAndFinalization 17.242 us/op 19.786 us/op 0.87
mainnet_e58758 - phase0 processRewardsAndPenalties 34.493 ms/op 34.784 ms/op 0.99
mainnet_e58758 - phase0 processRegistryUpdates 8.4730 us/op 9.6730 us/op 0.88
mainnet_e58758 - phase0 processSlashings 400.00 ns/op 869.00 ns/op 0.46
mainnet_e58758 - phase0 processEth1DataReset 430.00 ns/op 747.00 ns/op 0.58
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.3655 ms/op 814.30 us/op 1.68
mainnet_e58758 - phase0 processSlashingsReset 3.6440 us/op 5.2260 us/op 0.70
mainnet_e58758 - phase0 processRandaoMixesReset 4.3010 us/op 3.6550 us/op 1.18
mainnet_e58758 - phase0 processHistoricalRootsUpdate 465.00 ns/op 846.00 ns/op 0.55
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.6180 us/op 3.2040 us/op 1.44
mainnet_e58758 - phase0 afterProcessEpoch 45.526 ms/op 36.038 ms/op 1.26
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.9387 ms/op 1.0017 ms/op 1.94
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.2522 ms/op 1.8746 ms/op 1.20
altair processInactivityUpdates - 250000 normalcase 16.074 ms/op 17.320 ms/op 0.93
altair processInactivityUpdates - 250000 worstcase 16.831 ms/op 16.996 ms/op 0.99
phase0 processRegistryUpdates - 250000 normalcase 8.5550 us/op 9.2270 us/op 0.93
phase0 processRegistryUpdates - 250000 badcase_full_deposits 311.08 us/op 309.50 us/op 1.01
phase0 processRegistryUpdates - 250000 worstcase 0.5 128.29 ms/op 110.84 ms/op 1.16
altair processRewardsAndPenalties - 250000 normalcase 49.936 ms/op 34.267 ms/op 1.46
altair processRewardsAndPenalties - 250000 worstcase 45.671 ms/op 43.353 ms/op 1.05
phase0 getAttestationDeltas - 250000 normalcase 9.3319 ms/op 6.3054 ms/op 1.48
phase0 getAttestationDeltas - 250000 worstcase 8.1051 ms/op 6.2771 ms/op 1.29
phase0 processSlashings - 250000 worstcase 129.25 us/op 95.435 us/op 1.35
altair processSyncCommitteeUpdates - 250000 141.29 ms/op 104.39 ms/op 1.35
BeaconState.hashTreeRoot - No change 258.00 ns/op 442.00 ns/op 0.58
BeaconState.hashTreeRoot - 1 full validator 138.62 us/op 144.94 us/op 0.96
BeaconState.hashTreeRoot - 32 full validator 1.5960 ms/op 1.4787 ms/op 1.08
BeaconState.hashTreeRoot - 512 full validator 11.941 ms/op 10.679 ms/op 1.12
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 143.28 us/op 142.06 us/op 1.01
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.5193 ms/op 1.8790 ms/op 0.81
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 19.038 ms/op 22.012 ms/op 0.86
BeaconState.hashTreeRoot - 1 balances 101.54 us/op 106.58 us/op 0.95
BeaconState.hashTreeRoot - 32 balances 824.84 us/op 1.0473 ms/op 0.79
BeaconState.hashTreeRoot - 512 balances 7.4301 ms/op 7.7566 ms/op 0.96
BeaconState.hashTreeRoot - 250000 balances 180.94 ms/op 129.55 ms/op 1.40
aggregationBits - 2048 els - zipIndexesInBitList 25.904 us/op 50.929 us/op 0.51
byteArrayEquals 32 56.914 ns/op 47.742 ns/op 1.19
Buffer.compare 32 18.023 ns/op 15.808 ns/op 1.14
byteArrayEquals 1024 1.6878 us/op 1.2760 us/op 1.32
Buffer.compare 1024 28.322 ns/op 24.547 ns/op 1.15
byteArrayEquals 16384 27.518 us/op 20.198 us/op 1.36
Buffer.compare 16384 216.60 ns/op 187.46 ns/op 1.16
byteArrayEquals 123687377 198.88 ms/op 146.16 ms/op 1.36
Buffer.compare 123687377 7.6019 ms/op 5.5138 ms/op 1.38
byteArrayEquals 32 - diff last byte 53.905 ns/op 46.076 ns/op 1.17
Buffer.compare 32 - diff last byte 17.810 ns/op 15.199 ns/op 1.17
byteArrayEquals 1024 - diff last byte 1.6366 us/op 1.2391 us/op 1.32
Buffer.compare 1024 - diff last byte 25.907 ns/op 22.217 ns/op 1.17
byteArrayEquals 16384 - diff last byte 25.740 us/op 19.368 us/op 1.33
Buffer.compare 16384 - diff last byte 207.30 ns/op 187.16 ns/op 1.11
byteArrayEquals 123687377 - diff last byte 197.07 ms/op 147.14 ms/op 1.34
Buffer.compare 123687377 - diff last byte 7.4639 ms/op 5.3933 ms/op 1.38
byteArrayEquals 32 - random bytes 5.5660 ns/op 4.7700 ns/op 1.17
Buffer.compare 32 - random bytes 17.771 ns/op 15.169 ns/op 1.17
byteArrayEquals 1024 - random bytes 5.3670 ns/op 4.7960 ns/op 1.12
Buffer.compare 1024 - random bytes 17.881 ns/op 14.900 ns/op 1.20
byteArrayEquals 16384 - random bytes 5.3430 ns/op 4.8320 ns/op 1.11
Buffer.compare 16384 - random bytes 17.930 ns/op 14.986 ns/op 1.20
byteArrayEquals 123687377 - random bytes 6.6400 ns/op 7.5900 ns/op 0.87
Buffer.compare 123687377 - random bytes 19.350 ns/op 17.710 ns/op 1.09
regular array get 100000 times 44.371 us/op 29.810 us/op 1.49
wrappedArray get 100000 times 37.901 us/op 29.748 us/op 1.27
arrayWithProxy get 100000 times 13.326 ms/op 9.6938 ms/op 1.37
ssz.Root.equals 46.380 ns/op 38.378 ns/op 1.21
byteArrayEquals 45.555 ns/op 41.944 ns/op 1.09
Buffer.compare 10.523 ns/op 8.7390 ns/op 1.20
processSlot - 1 slots 11.745 us/op 12.137 us/op 0.97
processSlot - 32 slots 2.2378 ms/op 2.0926 ms/op 1.07
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 37.784 ms/op 38.011 ms/op 0.99
getCommitteeAssignments - req 1 vs - 250000 vc 2.2854 ms/op 1.8038 ms/op 1.27
getCommitteeAssignments - req 100 vs - 250000 vc 4.3384 ms/op 3.5153 ms/op 1.23
getCommitteeAssignments - req 1000 vs - 250000 vc 4.6474 ms/op 3.7587 ms/op 1.24
findModifiedValidators - 10000 modified validators 280.65 ms/op 241.54 ms/op 1.16
findModifiedValidators - 1000 modified validators 202.00 ms/op 143.46 ms/op 1.41
findModifiedValidators - 100 modified validators 172.02 ms/op 135.70 ms/op 1.27
findModifiedValidators - 10 modified validators 183.03 ms/op 144.80 ms/op 1.26
findModifiedValidators - 1 modified validators 161.77 ms/op 135.00 ms/op 1.20
findModifiedValidators - no difference 182.61 ms/op 145.99 ms/op 1.25
compare ViewDUs 3.3910 s/op 3.0838 s/op 1.10
compare each validator Uint8Array 1.1742 s/op 1.1173 s/op 1.05
compare ViewDU to Uint8Array 1.1845 s/op 753.23 ms/op 1.57
migrate state 1000000 validators, 24 modified, 0 new 792.66 ms/op 704.54 ms/op 1.13
migrate state 1000000 validators, 1700 modified, 1000 new 1.1447 s/op 886.77 ms/op 1.29
migrate state 1000000 validators, 3400 modified, 2000 new 1.3740 s/op 1.1221 s/op 1.22
migrate state 1500000 validators, 24 modified, 0 new 803.66 ms/op 662.53 ms/op 1.21
migrate state 1500000 validators, 1700 modified, 1000 new 1.0229 s/op 911.90 ms/op 1.12
migrate state 1500000 validators, 3400 modified, 2000 new 1.1536 s/op 1.1103 s/op 1.04
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.9900 ns/op 6.1000 ns/op 0.82
state getBlockRootAtSlot - 250000 vs - 7PWei 472.75 ns/op 376.82 ns/op 1.25
computeProposers - vc 250000 6.8902 ms/op 5.4958 ms/op 1.25
computeEpochShuffling - vc 250000 42.881 ms/op 35.092 ms/op 1.22
getNextSyncCommittee - vc 250000 133.98 ms/op 106.70 ms/op 1.26
computeSigningRoot for AttestationData 27.195 us/op 22.188 us/op 1.23
hash AttestationData serialized data then Buffer.toString(base64) 1.7385 us/op 1.1877 us/op 1.46
toHexString serialized data 1.0052 us/op 811.85 ns/op 1.24
Buffer.toString(base64) 204.10 ns/op 138.27 ns/op 1.48
nodejs block root to RootHex using toHex 171.27 ns/op 126.28 ns/op 1.36
nodejs block root to RootHex using toRootHex 100.85 ns/op 79.003 ns/op 1.28
browser block root to RootHex using the deprecated toHexString 240.04 ns/op 216.69 ns/op 1.11
browser block root to RootHex using toHex 192.82 ns/op 164.95 ns/op 1.17
browser block root to RootHex using toRootHex 173.27 ns/op 146.22 ns/op 1.18

by benchmarkbot/action

@twoeths
Copy link
Contributor Author

twoeths commented Nov 27, 2024

was able to sync with this branch from genesis (in progress, not finish yet), the error is gone

grep -e "Skip blobSidecars migration" -rn beacon-2024-11-27.log | wc -l
778

@twoeths twoeths marked this pull request as ready for review November 27, 2024 09:12
nflaig
nflaig approved these changes Nov 27, 2024
View reviewed changes
Copy link
Member

@nflaig nflaig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, without these changing we can't sync a node post-deneb if blobs are expired already without using checkpoint sync.

The sepolia node is syncing post-deneb without errors while on unstable branch it was just stuck on deneb fork transition.

info: Syncing - 4.2 days left - 5.77 slots/s - slot: 6413947 - head: 4308799 0x1ff2…e8c9 - exec-block: valid(5248314 0xe48b…) - finalized: 0xf150…04d4:134648 - peers: 15
info: Syncing - 4.3 days left - 5.65 slots/s - slot: 6413948 - head: 4308831 0xaf60…7e6b - exec-block: valid(5248342 0xa31d…) - finalized: 0xac66…0bf3:134649 - peers: 16
info: Syncing - 4.6 days left - 5.30 slots/s - slot: 6413949 - head: 4308863 0x42c8…7d97 - exec-block: valid(5248374 0x6b22…) - finalized: 0xfb0a…8d20:134650 - peers: 16

The Error processing finalized checkpoint error is no longer happening.

@twoeths twoeths merged commit 0c1ad93 into unstable Nov 27, 2024
20 checks passed
@twoeths twoeths deleted the te/sync_deneb_out_of_range_blob_block branch November 27, 2024 10:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nflaig nflaig nflaig approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Mekong: cannot sync from genesis
2 participants
@twoeths @nflaig