Faster MEV tip claiming (v1.17) #568

buffalu · 2024-01-21T00:41:56Z

Problem

MEV tip claiming has been slow and almost taking an epoch.

Summary of Changes

Gather all transactions, take a random subset of them, and send them all on-chain until the blockhash expires.
Periodically check to see which signatures landed.
Repeat until timeout or finished.
Run reclaim rent and claim mev in parallel. This should prevent us from needing to top up as frequently as reclaim rent should re-fill. One could even run reclaim rent first then run mev claim since there can be race conditions between claim mev + reclaim rent, but running in parallel seems more ideal

Note: I need to merge this upstream into master, but this makes it easier to work on

esemeniuc · 2024-01-23T06:09:16Z

tip-distributor/src/bin/claim-mev-tips.rs

+    let epoch = merkle_trees.epoch;
+
+    let mut futs = vec![];
+    futs.push(tokio::spawn(start_mev_claim_process(


a bit confused why spawning futures through join_all? tokio::spawn is non async

can you explain more? this is typical pattern we do throughout our own codebases

esemeniuc · 2024-01-23T06:11:34Z

tip-distributor/src/lib.rs

+) -> solana_rpc_client_api::client_error::Result<Vec<(Signature, Option<TransactionStatus>)>> {
+    let mut signature_statuses = Vec::new();
+
+    for signatures_batch in signatures.chunks(100) {


old code would request in parallel, any issues come up because of that?

esemeniuc · 2024-01-23T06:13:03Z

tip-distributor/src/claim_mev_workflow.rs

+        .collect();
+
+    datapoint_info!(
+        "build_transactions_new",


prefix with claim_mev_tips-...?

tip-distributor/src/lib.rs

tip-distributor/src/claim_mev_workflow.rs

tip-distributor/src/reclaim_rent_workflow.rs

esemeniuc · 2024-01-23T06:51:03Z

tip-distributor/src/bin/claim-mev-tips.rs

    solana_metrics::flush(); // sometimes last datapoint doesn't get emitted. this increases likelihood.
+    for r in results {
+        r.map_err(|e| ClaimMevError::UncaughtError { e: e.to_string() })??;


could just unwrap these directly?

main returns error, so just propagating that out

Be more aggressive about sending transactions.

Backport clone derivation (416) (#418) backports #430: update jito-programs (#432) Backport #446 to v1.17 (#448) [JIT-1661] Faster Autosnapshot (#445) v1.17: Backport #449 (#451) backports #419: add upsert to accountoverrides (#421) backport 428 runtime-plugin (#458) [JIT-1713] Fix bundle's blockspace preallocation (Backport to 1.17) (#499) Backport #500 to v1.17 (#507) Backport #520: add priority fees to mev claim (#525) Faster MEV tip claiming (v1.17) (#568)

Backport clone derivation (416) (#418) backports #430: update jito-programs (#432) Backport #446 to v1.17 (#448) [JIT-1661] Faster Autosnapshot (#445) v1.17: Backport #449 (#451) backports #419: add upsert to accountoverrides (#421) backport 428 runtime-plugin (#458) [JIT-1713] Fix bundle's blockspace preallocation (Backport to 1.17) (#499) Backport #500 to v1.17 (#507) Backport #520: add priority fees to mev claim (#525) Faster MEV tip claiming (v1.17) (#568) Backport Release Fixes (#586)

Backport clone derivation (416) (#418) backports #430: update jito-programs (#432) Backport #446 to v1.17 (#448) [JIT-1661] Faster Autosnapshot (#445) v1.17: Backport #449 (#451) backports #419: add upsert to accountoverrides (#421) backport 428 runtime-plugin (#458) [JIT-1713] Fix bundle's blockspace preallocation (Backport to 1.17) (#499) Backport #500 to v1.17 (#507) Backport #520: add priority fees to mev claim (#525) Faster MEV tip claiming (v1.17) (#568)

Backport clone derivation (416) (#418) backports #430: update jito-programs (#432) Backport #446 to v1.17 (#448) [JIT-1661] Faster Autosnapshot (#445) v1.17: Backport #449 (#451) backports #419: add upsert to accountoverrides (#421) backport 428 runtime-plugin (#458) [JIT-1713] Fix bundle's blockspace preallocation (Backport to 1.17) (#499) Backport #500 to v1.17 (#507) Backport #520: add priority fees to mev claim (#525) Faster MEV tip claiming (v1.17) (#568) Backport Release Fixes (#586)

Backport clone derivation (416) (jito-foundation#418) backports jito-foundation#430: update jito-programs (jito-foundation#432) Backport jito-foundation#446 to v1.17 (jito-foundation#448) [JIT-1661] Faster Autosnapshot (jito-foundation#445) v1.17: Backport jito-foundation#449 (jito-foundation#451) backports jito-foundation#419: add upsert to accountoverrides (jito-foundation#421) backport 428 runtime-plugin (jito-foundation#458) [JIT-1713] Fix bundle's blockspace preallocation (Backport to 1.17) (jito-foundation#499) Backport jito-foundation#500 to v1.17 (jito-foundation#507) Backport jito-foundation#520: add priority fees to mev claim (jito-foundation#525) Faster MEV tip claiming (v1.17) (jito-foundation#568) Backport Release Fixes (jito-foundation#586)

Backport clone derivation (416) (#418) backports #430: update jito-programs (#432) Backport #446 to v1.17 (#448) [JIT-1661] Faster Autosnapshot (#445) v1.17: Backport #449 (#451) backports #419: add upsert to accountoverrides (#421) backport 428 runtime-plugin (#458) [JIT-1713] Fix bundle's blockspace preallocation (Backport to 1.17) (#499) Backport #500 to v1.17 (#507) Backport #520: add priority fees to mev claim (#525) Faster MEV tip claiming (v1.17) (#568) Backport Release Fixes (#586)

Backport clone derivation (416) (#418) backports #430: update jito-programs (#432) Backport #446 to v1.17 (#448) [JIT-1661] Faster Autosnapshot (#445) v1.17: Backport #449 (#451) backports #419: add upsert to accountoverrides (#421) backport 428 runtime-plugin (#458) [JIT-1713] Fix bundle's blockspace preallocation (Backport to 1.17) (#499) Backport #500 to v1.17 (#507) Backport #520: add priority fees to mev claim (#525) Faster MEV tip claiming (v1.17) (#568) Backport Release Fixes (#586) Update release URLs for jito-solana (#589) Backports #595: correctly initialize account overrides (#599) Fix: Ensure set contact info to UDP port instead of QUIC (#601) Buffer bundles that exceed processing time and make the allowed processing time longer (#609)

Backport clone derivation (416) (#418) backports #430: update jito-programs (#432) Backport #446 to v1.17 (#448) [JIT-1661] Faster Autosnapshot (#445) v1.17: Backport #449 (#451) backports #419: add upsert to accountoverrides (#421) backport 428 runtime-plugin (#458) [JIT-1713] Fix bundle's blockspace preallocation (Backport to 1.17) (#499) Backport #500 to v1.17 (#507) Backport #520: add priority fees to mev claim (#525) Faster MEV tip claiming (v1.17) (#568) Backport Release Fixes (#586)

Backport clone derivation (416) (#418) backports #430: update jito-programs (#432) Backport #446 to v1.17 (#448) [JIT-1661] Faster Autosnapshot (#445) v1.17: Backport #449 (#451) backports #419: add upsert to accountoverrides (#421) backport 428 runtime-plugin (#458) [JIT-1713] Fix bundle's blockspace preallocation (Backport to 1.17) (#499) Backport #500 to v1.17 (#507) Backport #520: add priority fees to mev claim (#525) Faster MEV tip claiming (v1.17) (#568) Backport Release Fixes (#586) Update release URLs for jito-solana (#589) Backports #595: correctly initialize account overrides (#599) Fix: Ensure set contact info to UDP port instead of QUIC (#601) Buffer bundles that exceed processing time and make the allowed processing time longer (#609)

buffalu force-pushed the v1.17 branch from 980b9a8 to 11cd9ee Compare January 22, 2024 00:29

buffalu force-pushed the lb/faster_claim_117 branch from d6a7db3 to 5b8db4e Compare January 22, 2024 00:37

buffalu requested review from esemeniuc, jedleggett and segfaultdoc January 22, 2024 02:43