Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: Log archiver syncing #10428

Closed
spalladino opened this issue Dec 5, 2024 · 3 comments · Fixed by #10429 or #10442
Closed

chore: Log archiver syncing #10428

spalladino opened this issue Dec 5, 2024 · 3 comments · Fixed by #10429 or #10442
Assignees
@spalladino

Comments

@spalladino
Copy link
Collaborator

spalladino commented Dec 5, 2024 •
edited
Loading

Archiver should show what % complete it is when syncing the chain.
Or at least print something to confirm its still working on syncing and not just frozen/dead.

From @aminsammara
@spalladino spalladino self-assigned this Dec 5, 2024
This was referenced Dec 5, 2024
Merged
Merged
spalladino added a commit that referenced this issue Dec 6, 2024
@spalladino @just-mitch
chore: Clean up archiver logs (#10429)
4fcbc59 
- Logs reorgs as warn
- Logs every new L2 block downloaded as info
- Logs messages and contracts retrieved as verbose
- Logs everything else (including detailed block info) as debug

Fixes #10428 as we log one entry per block synced

---------

Co-authored-by: Mitch <[email protected]>
@spalladino
Copy link
Collaborator Author

Not yet fixed, as pointed out by @Maddiaa0

@spalladino spalladino reopened this Dec 6, 2024
@Maddiaa0
Copy link
Member

Maddiaa0 commented Dec 6, 2024 •
edited
Loading

duplicate issue: #10242

@spalladino
Copy link
Collaborator Author

Closing as duplicate of #10242

@spalladino spalladino closed this as not planned Won't fix, can't repro, duplicate, stale Dec 6, 2024
spalladino added a commit that referenced this issue Dec 6, 2024
@spalladino
chore: Batch archiver requests (#10442)
9443e8e 
Instead of downloading every block and keeping it in memory, download in
batches of (configurable) 100 blocks and store each batch as it is
retrieved. Also downloads blocks in parallel within the batch once their
logs have been retrieved.

Following is an `info` level logging for the archiver, with a batch size
of 5 (for testing) and an artifical delay of 1s for fetching logs.
Notice the timestamps:

```
[11:10:38.855] INFO: archiver Starting archiver sync to rollup contract 0x2b09a171300b3820c14f2faea559acc9270787de from L1 block 20 to current L1 block 265
[11:10:41.907] INFO: archiver Downloaded L2 block 2
    blockHash: "0x0d088ff05560afa67b8a1a9d7099310ba3268426dd4a488bcfdff6fdef59607f"
    blockNumber: 2
[11:10:41.907] INFO: archiver Downloaded L2 block 1
    blockHash: "0x150e8994ed59731daaed645e9e3d1687d7eb3648d08eca027f3dacf2deafdd71"
    blockNumber: 1
[11:10:43.957] INFO: archiver Downloaded L2 block 3
    blockHash: "0x1b0fe9562164a3e081dede1a47765be7f7f1d53842af58b4240686dad4270878"
    blockNumber: 3
[11:10:43.958] INFO: archiver Downloaded L2 block 5
    blockHash: "0x27be6c799c01cd97c203c5141deae0f9a1eb694fc5aaf57a06a90bff9d910916"
    blockNumber: 5
[11:10:43.959] INFO: archiver Downloaded L2 block 4
    blockHash: "0x22904cda410acba1ed66b180bbd0057b2a2cf61564267ac87ba77a237f961986"
    blockNumber: 4
[11:10:43.959] INFO: archiver Downloaded L2 block 6
    blockHash: "0x12857e4319f643ad58f7e2a95470b482b166897038d23263a97c277dbeb82b54"
    blockNumber: 6
[11:10:44.991] INFO: archiver Downloaded L2 block 7
    blockHash: "0x2e82579f660cae4a1c38c80f781d4dc03adf97aa8098b2d6e4157b56c79e415b"
    blockNumber: 7
[11:10:44.991] INFO: archiver Downloaded L2 block 9
    blockHash: "0x01848df8c72f6c51e23e5aa898bdae14a381ff4de7a28e592b0529e1a114a3af"
    blockNumber: 9
[11:10:44.991] INFO: archiver Downloaded L2 block 8
    blockHash: "0x29ca2d4898d1a540d46f45e9a20c029d6765167e031ff7d51701be109189f8ae"
    blockNumber: 8
[11:10:44.992] INFO: archiver Downloaded L2 block 10
    blockHash: "0x0b286a06ffef588628974f94c376829a51f36aa59668f766c1bc4896f0690e75"
    blockNumber: 10
[11:10:46.027] INFO: archiver Downloaded L2 block 11
    blockHash: "0x2a05ceb6fd3c9400a73480b4bf796dbdd8f46cbb35e37e1165e50beb514bb3a4"
    blockNumber: 11
[11:10:46.027] INFO: archiver Downloaded L2 block 14
    blockHash: "0x179ad2642bce681dabbf4a277ef2cd5e1f7b6812bd9fac9358a6346187b81eb6"
    blockNumber: 14
[11:10:46.027] INFO: archiver Downloaded L2 block 12
    blockHash: "0x2242c7eaa8133409069ca6c6f466b08672f0c3308a0ce89c4306908f1ebd2f4d"
    blockNumber: 12
[11:10:46.028] INFO: archiver Downloaded L2 block 13
    blockHash: "0x19070934599ca22124e158f29e3476fd69575773dc987cd93aa3dc25341c2ce0"
    blockNumber: 13
[11:10:47.060] INFO: archiver Downloaded L2 block 18
    blockHash: "0x0b3b73690622ab3110b633d306e77e7ba17dfc52727e47177f274211e85faa5c"
    blockNumber: 18
[11:10:47.060] INFO: archiver Downloaded L2 block 15
    blockHash: "0x1641ce6f8bf798f9e9adb91b74fc0f1a13b822156309e5c9b4139b705ca65854"
    blockNumber: 15
[11:10:47.060] INFO: archiver Downloaded L2 block 16
    blockHash: "0x11d802bf4c814458d463479420dc88fc5fa0ccd114bab910f23a8e028d6af907"
    blockNumber: 16
[11:10:47.061] INFO: archiver Downloaded L2 block 17
    blockHash: "0x2f16f9fe977fd0202b46b494f35a313353eb90eb91402925110422419ada04f3"
    blockNumber: 17
[11:10:48.094] INFO: archiver Downloaded L2 block 21
    blockHash: "0x2bd6a5e4433f87eb5290c0d9daef97ab0819137e67a1b83a2d232c95f887bac5"
    blockNumber: 21
[11:10:48.094] INFO: archiver Downloaded L2 block 22
    blockHash: "0x0fba866a869b819e181f95743047998e6f9d516fc4c40247e72ca76ecf6cafe3"
    blockNumber: 22
[11:10:48.094] INFO: archiver Downloaded L2 block 20
    blockHash: "0x168ca8474eef62fc291eb9c1e73973ffc2b820961d9285b00df3666d617b8f8e"
    blockNumber: 20
[11:10:48.095] INFO: archiver Downloaded L2 block 19
    blockHash: "0x1aa0fdd5d089aae60efb7c1f27d4dff86ba6c0d0d5021e2cec30e94d957df26d"
    blockNumber: 19
[11:10:49.127] INFO: archiver Downloaded L2 block 25
    blockHash: "0x00d65386773e5786f39883d9ce9905a17ab77abe30d1567bec168793ba5af1b8"
    blockNumber: 25
[11:10:49.128] INFO: archiver Downloaded L2 block 23
    blockHash: "0x26fe1e7196f76d6b4ae0342b878ec21525e44ab6b9038d8014039807decf6ab5"
    blockNumber: 23
[11:10:49.128] INFO: archiver Downloaded L2 block 26
    blockHash: "0x052ba994594f33ffa0a3801547351d8af8e1657888ff175381180b801041452f"
    blockNumber: 26
[11:10:49.128] INFO: archiver Downloaded L2 block 24
    blockHash: "0x086bacb80c17ca96234eb727e931195916f6fb5fb5b145d8f96559394dde9e56"
    blockNumber: 24
[11:10:49.129] INFO: archiver Downloaded L2 block 22
    blockHash: "0x0fba866a869b819e181f95743047998e6f9d516fc4c40247e72ca76ecf6cafe3"
    blockNumber: 22
[11:10:50.172] INFO: archiver Downloaded L2 block 30
    blockHash: "0x091c1f8bcaf206eae58921ff2859320ed9e7822a82a95ff99fcc60fca0614a42"
    blockNumber: 30
[11:10:50.173] INFO: archiver Downloaded L2 block 29
    blockHash: "0x2614dc5fc6408580f564b057fac6c3afa2d0e8fd2be428a3481d12ae89ade7e9"
    blockNumber: 29
[11:10:50.173] INFO: archiver Downloaded L2 block 27
    blockHash: "0x140a16edfa828dc786246cb3a834c3103538a19c79d21f6298c4090c49fa8e0e"
    blockNumber: 27
[11:10:50.173] INFO: archiver Downloaded L2 block 28
    blockHash: "0x1d7793e2df27dfad0f8a41bbb4d13e1701be133ecc117b2c8a2c02abb51dd513"
    blockNumber: 28
[11:10:50.173] INFO: archiver Downloaded L2 block 26
    blockHash: "0x052ba994594f33ffa0a3801547351d8af8e1657888ff175381180b801041452f"
    blockNumber: 26
[11:10:51.207] INFO: archiver Downloaded L2 block 30
    blockHash: "0x091c1f8bcaf206eae58921ff2859320ed9e7822a82a95ff99fcc60fca0614a42"
    blockNumber: 30
[11:10:51.207] INFO: archiver Downloaded L2 block 32
    blockHash: "0x204264d3fe9f82b61bb1405bb02cfc59299178dfdebeba75e0691597fd59c000"
    blockNumber: 32
[11:10:51.208] INFO: archiver Downloaded L2 block 34
    blockHash: "0x164d62ba59c01e9a29740d4c83368e15de8bb5a5b8621e8bdd1d0e0978e99d79"
    blockNumber: 34
[11:10:51.208] INFO: archiver Downloaded L2 block 33
    blockHash: "0x007dcb5015c13006cabda887a0e023899e5c327a340179dbd3ec56edc58e146a"
    blockNumber: 33
[11:10:51.208] INFO: archiver Downloaded L2 block 31
    blockHash: "0x00ad45cc4b2b7f0916bd86c11d12628321715b43e204662d15c2c174285ff608"
    blockNumber: 31
[11:10:53.242] INFO: archiver Downloaded L2 block 34
    blockHash: "0x164d62ba59c01e9a29740d4c83368e15de8bb5a5b8621e8bdd1d0e0978e99d79"
    blockNumber: 34
[11:10:53.243] INFO: archiver Downloaded L2 block 35
    blockHash: "0x19252e14d4fd8af5e47651cf31abaeb310a6a59c9754d642fba30ac6853fd076"
    blockNumber: 35
[11:10:53.244] INFO: archiver Downloaded L2 block 37
    blockHash: "0x17ee8d4ba85eb5b2f6a0bcb70115bdd94c90c1dc8fc89f1c9d2418ab944395ac"
    blockNumber: 37
[11:10:53.244] INFO: archiver Downloaded L2 block 36
    blockHash: "0x0b0b0f4694fe81717cb30f59f5dc5d6dc5b632c25832ac424eea074d0c3c19a7"
    blockNumber: 36
[11:10:55.285] INFO: archiver Downloaded L2 block 41
    blockHash: "0x2f7f979d0bae806fcd0bfaed001c1286ae569ecf85aa4063b0906f2097dd5c4c"
    blockNumber: 41
[11:10:55.285] INFO: archiver Downloaded L2 block 38
    blockHash: "0x016418f770c61aa2e649433ab59c2a124acb8e10ac395bb3e1f8dbf84f405039"
    blockNumber: 38
[11:10:55.286] INFO: archiver Downloaded L2 block 39
    blockHash: "0x1588ad911757f334ad2e969588f7fe149bde4a49c763c636af0c6ffb4f475660"
    blockNumber: 39
[11:10:55.287] INFO: archiver Downloaded L2 block 40
    blockHash: "0x06e69f6ba327f6acb88de656058f68eb4998dbee0fec04cf891aabf3b047b071"
    blockNumber: 40
[11:10:57.327] INFO: archiver Downloaded L2 block 42
    blockHash: "0x1ea461af6e875e1bb91dd729f196245ac54b2652799172bba10f77fa2a2fb248"
    blockNumber: 42
[11:10:57.328] INFO: archiver Downloaded L2 block 45
    blockHash: "0x1f7e13643d2286e77dc8de2e37c73d230b77f4cbd7e11fdb9eb7b1c136fe180a"
    blockNumber: 45
[11:10:57.328] INFO: archiver Downloaded L2 block 43
    blockHash: "0x0d89250055f022310144c869ecfdf8da7099fd9bd5e5680687a64b8ad7e588ce"
    blockNumber: 43
[11:10:57.329] INFO: archiver Downloaded L2 block 44
    blockHash: "0x2212d3ae031d4549789c39a403f91f4f06e4e031244700dbb56b381d7730c203"
    blockNumber: 44
[11:10:59.363] INFO: archiver Downloaded L2 block 46
    blockHash: "0x04028bca36e3e6644ceef2425c636ba594e73f4e50d673766e6d4ce8e8b256fb"
    blockNumber: 46
[11:10:59.364] INFO: archiver Downloaded L2 block 49
    blockHash: "0x035b4fc20e48c41bc850e3a4c3c17f4d2004ef3704c660275b63c04925a7e139"
    blockNumber: 49
[11:10:59.364] INFO: archiver Downloaded L2 block 48
    blockHash: "0x2993838786ed7f76d994e9e86f8801657666857b2e46ba9c874c9512a701d112"
    blockNumber: 48
[11:10:59.365] INFO: archiver Downloaded L2 block 47
    blockHash: "0x075078e4ad9c68625585d02bc501bf5645c7cd6601c91090daa88915c8c97a8d"
    blockNumber: 47
[11:11:00.405] INFO: archiver Downloaded L2 block 53
    blockHash: "0x17bd954fbe9ed7c3c376fb340c302dd97297284ebc5c3a5c65f1693a5b26fa0a"
    blockNumber: 53
[11:11:00.405] INFO: archiver Downloaded L2 block 50
    blockHash: "0x0c95eeb66fa8e7704940d81faa1f6d084bc38dc7ffd7a697f1344c50507088dc"
    blockNumber: 50
[11:11:00.406] INFO: archiver Downloaded L2 block 52
    blockHash: "0x2841bc4f270bbc36e31336bbb1e22d90de24134beb16568eda8352b80d364b3b"
    blockNumber: 52
[11:11:00.406] INFO: archiver Downloaded L2 block 51
    blockHash: "0x2dd14899c2e30294e35e795d267a36890f5971d2e265e59645ed5df9acf1b336"
    blockNumber: 51
[11:11:01.460] INFO: archiver Downloaded L2 block 57
    blockHash: "0x1b55c1c5375a29d7f5793c19af6fbf0a5b8183eab97695298e0b1eafd61693b5"
    blockNumber: 57
[11:11:01.461] INFO: archiver Downloaded L2 block 55
    blockHash: "0x19635ee907e378546db1a15810db50b5e77f526d68a19040c482d5ccefb0127e"
    blockNumber: 55
[11:11:01.461] INFO: archiver Downloaded L2 block 56
    blockHash: "0x0847cee0bce17dface027efbb368ac35773f372fef20d9a4afa0947f216ac20d"
    blockNumber: 56
[11:11:01.463] INFO: archiver Downloaded L2 block 54
    blockHash: "0x29043b57317ba3f06994a6638d2185b88b0b1f868e4ccbaf510671a951a3f424"
    blockNumber: 54
[11:11:02.517] INFO: archiver Downloaded L2 block 58
    blockHash: "0x00e95202a2906628cba2c0061519b064757b7a7884e6a8a584b08cfc3baf3825"
    blockNumber: 58
[11:11:02.518] INFO: archiver Downloaded L2 block 61
    blockHash: "0x1b3425cb6f5e199f6f5e0848d3f741a8980fef5fb5fff3a142163c460726dfa9"
    blockNumber: 61
[11:11:02.518] INFO: archiver Downloaded L2 block 59
    blockHash: "0x1e80799672dbfd1fad11e0607cc063e9c0749e8f8b82474f0d579ceb66d5db91"
    blockNumber: 59
[11:11:02.519] INFO: archiver Downloaded L2 block 60
    blockHash: "0x2212c79084a2793836b5fdf446bdd2c4def00d68ec64ebae90b64a16664c6b30"
    blockNumber: 60
[11:11:03.572] INFO: archiver Downloaded L2 block 62
    blockHash: "0x0fd061e3ee9e44a697d4fe81160897cda6df121937b385db7d0b24900ef62a23"
    blockNumber: 62
[11:11:03.573] INFO: archiver Downloaded L2 block 63
    blockHash: "0x2dffde0e911c1f41427d1122cbf6dd28823560c826381094c7f6f26c01f97271"
    blockNumber: 63
[11:11:03.574] INFO: archiver Downloaded L2 block 65
    blockHash: "0x228f48d523eafdc673152ae8e9510eaf7bb856390a0c3d43d8cca97eda0ed217"
    blockNumber: 65
[11:11:03.574] INFO: archiver Downloaded L2 block 64
    blockHash: "0x2bcd0a71d39f8d2e5a290608d8c719e20b264e62227c68152fcd5541f753e25e"
    blockNumber: 64
[11:11:04.638] INFO: archiver Downloaded L2 block 69
    blockHash: "0x1cb98870e6ffbab13afb58add2b024ef29cf9345cced66cd8f0f8f1e0e675ac1"
    blockNumber: 69
[11:11:04.639] INFO: archiver Downloaded L2 block 67
    blockHash: "0x03c0de0563d63f8365e58d34f3ff55f50c85cafdbf97d31b4e698be2720483b9"
    blockNumber: 67
[11:11:04.640] INFO: archiver Downloaded L2 block 66
    blockHash: "0x164d1b0a7ff82e959330bb7a9af22e9f3bf9aa85351891a108cd87586fba048d"
    blockNumber: 66
[11:11:04.640] INFO: archiver Downloaded L2 block 68
    blockHash: "0x0aa4502055e2ee2f44548ece76c8829fdf79b5a514bf40ae238ff62685f8a23b"
    blockNumber: 68
[11:11:04.641] INFO: archiver Downloaded L2 block 65
    blockHash: "0x228f48d523eafdc673152ae8e9510eaf7bb856390a0c3d43d8cca97eda0ed217"
    blockNumber: 65
[11:11:05.672] INFO: archiver Downloaded L2 block 71
    blockHash: "0x2b86465312009ffea5dbc436e97b47e11817068520b3d7f04c1117fe6e969630"
    blockNumber: 71
[11:11:05.672] INFO: archiver Downloaded L2 block 70
    blockHash: "0x2951e898e112dcde4d91d1ca32ef23522baed0aa064a53864f7ea87770b44b32"
    blockNumber: 70
[11:11:05.672] INFO: archiver Downloaded L2 block 69
    blockHash: "0x1cb98870e6ffbab13afb58add2b024ef29cf9345cced66cd8f0f8f1e0e675ac1"
    blockNumber: 69
[11:11:05.673] INFO: archiver Downloaded L2 block 73
    blockHash: "0x2c95db4f54323cb8ef5e9668570752a237b3b176c9d298e3cae650909593514d"
    blockNumber: 73
[11:11:05.673] INFO: archiver Downloaded L2 block 72
    blockHash: "0x210e66af85dc45471729d1ed625eef96ece7a15f54a0dfb18aac0ee35721ec1c"
    blockNumber: 72
[11:11:06.721] INFO: archiver Downloaded L2 block 77
    blockHash: "0x15f71bdd862b0d0016d1001a70b912cf32321e7347e02c3827a64f1bee139922"
    blockNumber: 77
[11:11:06.721] INFO: archiver Downloaded L2 block 73
    blockHash: "0x2c95db4f54323cb8ef5e9668570752a237b3b176c9d298e3cae650909593514d"
    blockNumber: 73
[11:11:06.722] INFO: archiver Downloaded L2 block 74
    blockHash: "0x29fda4905834eb581d02541456bb48653d29e89790ab1203a348cb0d663d4636"
    blockNumber: 74
[11:11:06.722] INFO: archiver Downloaded L2 block 75
    blockHash: "0x1f4118d232a014a3c7776cac40fd1dc1baf219e811726002f715e9abe1120f09"
    blockNumber: 75
[11:11:06.722] INFO: archiver Downloaded L2 block 76
    blockHash: "0x0342ab801e16f9146b109adb704b3d442b45877bf4cf0f38ecbe65e53e9ff447"
    blockNumber: 76
[11:11:07.767] INFO: archiver Downloaded L2 block 77
    blockHash: "0x15f71bdd862b0d0016d1001a70b912cf32321e7347e02c3827a64f1bee139922"
    blockNumber: 77
[11:11:07.767] INFO: archiver Downloaded L2 block 79
    blockHash: "0x1897627bbe054c2dbf1c19062c50b4bab839fc4f119d0683af625c9ab6eef20a"
    blockNumber: 79
[11:11:07.767] INFO: archiver Downloaded L2 block 81
    blockHash: "0x0f502ab8cdb5ab5dcc5dd194e53775c4e512ffdd0645ea363ab52921072c2003"
    blockNumber: 81
[11:11:07.768] INFO: archiver Downloaded L2 block 78
    blockHash: "0x263a83bf5c2410135a08fb48b0323fad959742c5d3d0376b11c55009e0961003"
    blockNumber: 78
[11:11:07.768] INFO: archiver Downloaded L2 block 80
    blockHash: "0x15669498212e68da4e015a4eb90fe1b9404f7e33fa922b242b2c3982c2ac975a"
    blockNumber: 80
[11:11:08.806] INFO: archiver Downloaded L2 block 81
    blockHash: "0x0f502ab8cdb5ab5dcc5dd194e53775c4e512ffdd0645ea363ab52921072c2003"
    blockNumber: 81
[11:11:08.807] INFO: archiver Downloaded L2 block 82
    blockHash: "0x24f6959cf5f228e97ff2934de0b68ee6bdd4b0bc9cef9f6fb60da7e60b354dfb"
    blockNumber: 82
[11:11:08.811] INFO: archiver Updated proven chain to block 81 (epoch 7)
    provenBlockNumber: 81
    provenEpochNumber: 7
[11:11:08.813] INFO: archiver Initial archiver sync to L1 block 265 complete.
```

Fixes #10428

Fixes #10242
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@spalladino spalladino

Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@spalladino @Maddiaa0