From d00c50d75ba4df6ebc0d9454aecf4cffc13f9cc3 Mon Sep 17 00:00:00 2001 From: teor Date: Thu, 17 Feb 2022 12:15:30 +1000 Subject: [PATCH 1/3] fix(rfc): rename some lightwalletd database types ``` fastmod BlockTransactionCount HeightTransactionCount fastmod FirstOutLocation TransparentAddrLoc ``` --- book/src/dev/rfcs/0005-state-updates.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/book/src/dev/rfcs/0005-state-updates.md b/book/src/dev/rfcs/0005-state-updates.md index 5866eb136f9..b16750fdc7e 100644 --- a/book/src/dev/rfcs/0005-state-updates.md +++ b/book/src/dev/rfcs/0005-state-updates.md @@ -603,15 +603,15 @@ We use the following rocksdb column families: | Column Family | Keys | Values | Updates | | ------------------------------ | ---------------------- | ----------------------------------- | ------- | | `hash_by_height` | `block::Height` | `block::Hash` | Never | -| `height_tx_count_by_hash` | `block::Hash` | `BlockTransactionCount` | Never | +| `height_tx_count_by_hash` | `block::Hash` | `HeightTransactionCount` | Never | | `block_header_by_height` | `block::Height` | `block::Header` | Never | | `tx_by_location` | `TransactionLocation` | `Transaction` | Never | | `hash_by_tx` | `TransactionLocation` | `transaction::Hash` | Never | | `tx_by_hash` | `transaction::Hash` | `TransactionLocation` | Never | | `utxo_by_outpoint` | `OutLocation` | `transparent::Output` | Delete | -| `balance_by_transparent_addr` | `transparent::Address` | `Amount \|\| FirstOutLocation` | Update | -| `utxo_by_transparent_addr_loc` | `FirstOutLocation` | `AtLeastOne` | Up/Del | -| `tx_by_transparent_addr_loc` | `FirstOutLocation` | `AtLeastOne` | Append | +| `balance_by_transparent_addr` | `transparent::Address` | `Amount \|\| TransparentAddrLoc` | Update | +| `utxo_by_transparent_addr_loc` | `TransparentAddrLoc` | `AtLeastOne` | Up/Del | +| `tx_by_transparent_addr_loc` | `TransparentAddrLoc` | `AtLeastOne` | Append | | `sprout_nullifiers` | `sprout::Nullifier` | `()` | Never | | `sprout_anchors` | `sprout::tree::Root` | `sprout::tree::NoteCommitmentTree` | Never | | `sprout_note_commitment_tree` | `block::Height` | `sprout::tree::NoteCommitmentTree` | Delete | @@ -636,7 +636,7 @@ Block and Transaction Data: - `TransparentOutputIndex`: 24 bits, big-endian, unsigned (max ~223,000 transfers in the 2 MB block limit) - transparent and shielded input indexes, and shielded output indexes: 16 bits, big-endian, unsigned (max ~49,000 transfers in the 2 MB block limit) - `OutLocation`: `TransactionLocation \|\| TransparentOutputIndex` -- `FirstOutLocation`: the first `OutLocation` used by a `transparent::Address`. +- `TransparentAddrLoc`: the first `OutLocation` used by a `transparent::Address`. Always has the same value for each address, even if the first output is spent. - `Utxo`: `Output`, derives extra fields from the `OutLocation` key - `AtLeastOne`: `[T; AtLeastOne::len()]` (for known-size `T`) @@ -737,7 +737,7 @@ So they should not be used for consensus-critical checks. - `balance_by_transparent_addr` is the sum of all `utxo_by_transparent_addr_loc`s that are still in `utxo_by_outpoint`. It is cached to improve performance for - addresses with large UTXO sets. It also stores the `FirstOutLocation` for each + addresses with large UTXO sets. It also stores the `TransparentAddrLoc` for each address, which allows for efficient lookups. - `utxo_by_transparent_addr_loc` stores unspent transparent output locations by address. @@ -745,12 +745,12 @@ So they should not be used for consensus-critical checks. has been spent in `utxo_by_outpoint`, that UTXO location can be deleted from `utxo_by_transparent_addr_loc`. (We don't do these deletions every time a block is committed, because that requires an expensive full index search.) - This list includes the `FirstOutLocation`, if it has not been spent. + This list includes the `TransparentAddrLoc`, if it has not been spent. (This duplicate data is small, and helps simplify the code.) - `tx_by_transparent_addr_loc` stores transaction locations by address. This list includes transactions containing spent UTXOs. - It also includes the `TransactionLocation` from the `FirstOutLocation`. + It also includes the `TransactionLocation` from the `TransparentAddrLoc`. (This duplicate data is small, and helps simplify the code.) - Each `*_note_commitment_tree` stores the note commitment tree state From f86d06826e80e2f209bc92d63ad1d81617d229ca Mon Sep 17 00:00:00 2001 From: teor Date: Thu, 17 Feb 2022 12:33:37 +1000 Subject: [PATCH 2/3] fix(rfc): make lightwalletd column family names consistent --- book/src/dev/rfcs/0005-state-updates.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/book/src/dev/rfcs/0005-state-updates.md b/book/src/dev/rfcs/0005-state-updates.md index b16750fdc7e..ba7871629c9 100644 --- a/book/src/dev/rfcs/0005-state-updates.md +++ b/book/src/dev/rfcs/0005-state-updates.md @@ -605,10 +605,10 @@ We use the following rocksdb column families: | `hash_by_height` | `block::Height` | `block::Hash` | Never | | `height_tx_count_by_hash` | `block::Hash` | `HeightTransactionCount` | Never | | `block_header_by_height` | `block::Height` | `block::Header` | Never | -| `tx_by_location` | `TransactionLocation` | `Transaction` | Never | -| `hash_by_tx` | `TransactionLocation` | `transaction::Hash` | Never | -| `tx_by_hash` | `transaction::Hash` | `TransactionLocation` | Never | -| `utxo_by_outpoint` | `OutLocation` | `transparent::Output` | Delete | +| `tx_by_loc` | `TransactionLocation` | `Transaction` | Never | +| `hash_by_tx_loc` | `TransactionLocation` | `transaction::Hash` | Never | +| `tx_loc_by_hash` | `transaction::Hash` | `TransactionLocation` | Never | +| `utxo_by_out_loc` | `OutLocation` | `transparent::Output` | Delete | | `balance_by_transparent_addr` | `transparent::Address` | `Amount \|\| TransparentAddrLoc` | Update | | `utxo_by_transparent_addr_loc` | `TransparentAddrLoc` | `AtLeastOne` | Up/Del | | `tx_by_transparent_addr_loc` | `TransparentAddrLoc` | `AtLeastOne` | Append | From f33e228f03f1967a46d7b686273882309cee0285 Mon Sep 17 00:00:00 2001 From: teor Date: Thu, 17 Feb 2022 12:38:49 +1000 Subject: [PATCH 3/3] feat(rfc): add sub-headings to the RocksDB column family list --- book/src/dev/rfcs/0005-state-updates.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/book/src/dev/rfcs/0005-state-updates.md b/book/src/dev/rfcs/0005-state-updates.md index ba7871629c9..f01d9674002 100644 --- a/book/src/dev/rfcs/0005-state-updates.md +++ b/book/src/dev/rfcs/0005-state-updates.md @@ -602,25 +602,32 @@ We use the following rocksdb column families: | Column Family | Keys | Values | Updates | | ------------------------------ | ---------------------- | ----------------------------------- | ------- | +| *Blocks* | | | | | `hash_by_height` | `block::Height` | `block::Hash` | Never | | `height_tx_count_by_hash` | `block::Hash` | `HeightTransactionCount` | Never | | `block_header_by_height` | `block::Height` | `block::Header` | Never | +| *Transactions* | | | | | `tx_by_loc` | `TransactionLocation` | `Transaction` | Never | | `hash_by_tx_loc` | `TransactionLocation` | `transaction::Hash` | Never | | `tx_loc_by_hash` | `transaction::Hash` | `TransactionLocation` | Never | +| *Transparent* | | | | | `utxo_by_out_loc` | `OutLocation` | `transparent::Output` | Delete | | `balance_by_transparent_addr` | `transparent::Address` | `Amount \|\| TransparentAddrLoc` | Update | | `utxo_by_transparent_addr_loc` | `TransparentAddrLoc` | `AtLeastOne` | Up/Del | | `tx_by_transparent_addr_loc` | `TransparentAddrLoc` | `AtLeastOne` | Append | +| *Sprout* | | | | | `sprout_nullifiers` | `sprout::Nullifier` | `()` | Never | | `sprout_anchors` | `sprout::tree::Root` | `sprout::tree::NoteCommitmentTree` | Never | | `sprout_note_commitment_tree` | `block::Height` | `sprout::tree::NoteCommitmentTree` | Delete | +| *Sapling* | | | | | `sapling_nullifiers` | `sapling::Nullifier` | `()` | Never | | `sapling_anchors` | `sapling::tree::Root` | `()` | Never | | `sapling_note_commitment_tree` | `block::Height` | `sapling::tree::NoteCommitmentTree` | Delete | +| *Orchard* | | | | | `orchard_nullifiers` | `orchard::Nullifier` | `()` | Never | | `orchard_anchors` | `orchard::tree::Root` | `()` | Never | | `orchard_note_commitment_tree` | `block::Height` | `orchard::tree::NoteCommitmentTree` | Delete | +| *Chain* | | | | | `history_tree` | `block::Height` | `NonEmptyHistoryTree` | Delete | | `tip_chain_value_pool` | `()` | `ValueBalance` | Update |