Implements an internal txindex so we don't have to run bitcoind with txindex=1
#132
+790
−535
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sr-gi
changed the title
txindextxindex=1
sr-gi
changed the title
txindex=1txindex=1
sr-gi
force-pushed
the
130-txindex
branch
2 times, most recently
from
fc04b8e to
58ede34
Compare
sr-gi
added
feature request
teos/src/tx_index.rs
Outdated
| } | ||
|
|
||
| /// Fixes the index by removing disconnected data. | ||
| fn fix(&mut self, header: &BlockHeader) { |
There was a problem hiding this comment.
Wondering if this function name could be a bit more descriptive, like "remove_block" or something
There was a problem hiding this comment.
How about remove_disconnected_block?
mariocynicys
approved these changes
Oct 28, 2022
|
@mariocynicys added fixes from your review comments. @orbitalturtle did you have the chance to check the changes from your suggestions? |
|
@orbitalturtle @mariocynicys I will rebase the changes and merge this if you're happy with it |
Sorry I missed this. Let me quickly review this. |
|
@sr-gi I can take a second look this evening! |
orbitalturtle
approved these changes
Nov 4, 2022
In order to fix talaia-labs#130 we need to implement our own txindex. Turns out this is almost identical to our `LocatorCache`, so we can generalize it and use it for both purposes.
|
Rebased and squashed |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Generalizes the
Watcher'sLocatorCacheinto aTxIndexthat can be used both by theWatcherand theResponder. The former does use it in the same way as before (as aLocatorCacheto keep track of theLocatorsof the last n blocks). As for the latter, it is used to implement a pruned transaction index (aTxid:BlockHashmap).By adding this
TxIndexin theResponderwe don't have to runbitcoindwithtxindex=1anymore and, therefore, we are able to reduce the initial storage requirements significantly: from ~420 GB on disk at the time of writing to a few dozen KB in memory.With this new approach queries to past transactions will be performed internally instead of against
bitcoindin most cases. If a transaction cannot be found in our index nor in mempool it'll mean that it was confirmed long ago (confirmation count > ourTxIndexlength). TheResponderindex is set to beIRREVOCABLY_RESOLVEDblocks long, meaning that if we don't find a transaction on it we don't need to care anyway.