From 328b674edc293a0cf090ebacb1d739d52ac85544 Mon Sep 17 00:00:00 2001 From: steviez Date: Mon, 13 Feb 2023 17:33:24 -0600 Subject: [PATCH] Remove recursive read lock that could deadlock Blockstore (#30203) This deadlock could only occur on nodes that call Blockstore::get_rooted_block(). Regular validators don't call this function, RPC nodes and nodes that have BigTableUploadService enabled do. Blockstore::get_rooted_block() grabs a read lock on lowest_cleanup_slot right at the start to check if the block has been cleaned up, and to ensure it doesn't get cleaned up during execution. As part of the callstack of get_rooted_block(), Blockstore::get_completed_ranges() will get called, which also grabs a read lock on lowest_cleanup_slot. If LedgerCleanupService attempts to grab a write lock between the read lock calls, we could hit a deadlock if priority is given to the write lock request in this scenario. This change removes the call to get the read lock in get_completed_ranges(). The lock is only held for the scope of this function, which is a single rocksdb read and thus not needed. This does mean that a different error will be returned if the requested slot was below lowest_cleanup_slot. Previously, a BlockstoreError::SlotCleanedUp would have been thrown; the rocksdb error will be bubbled up now. Note that callers of get_rooted_block() will still get the SlotCleanedUp error when appropriate because get_rooted_block() grabs the lock. If the slot is unavailable, it will return immediately. If the slot is available, get_rooted_block() holding the lock means the slot will remain available. --- ledger/src/blockstore.rs | 2 -- 1 file changed, 2 deletions(-) diff --git a/ledger/src/blockstore.rs b/ledger/src/blockstore.rs index 7ded4c83ce1be4..100b5149100f04 100644 --- a/ledger/src/blockstore.rs +++ b/ledger/src/blockstore.rs @@ -2900,8 +2900,6 @@ impl Blockstore { slot: Slot, start_index: u64, ) -> Result<(CompletedRanges, Option)> { - let _lock = self.check_lowest_cleanup_slot(slot)?; - let slot_meta_cf = self.db.column::(); let slot_meta = slot_meta_cf.get(slot)?; if slot_meta.is_none() {