feat(lib/trie): offload > 32B subvalues to database #2918
Conversation
qdm12
changed the title
qdm12
changed the base branch from
development
to
qdm12/trie/node-hashes-naming
qdm12
force-pushed
the
qdm12/trie/node-hashes-naming
branch
2 times, most recently
from
c20db3b to
1d957d2
Compare
qdm12
force-pushed
the
qdm12/trie/subvalue-disk
branch
from
c326ad4 to
3261e9f
Compare
qdm12
force-pushed
the
qdm12/trie/node-hashes-naming
branch
3 times, most recently
from
48346ce to
9a3c287
Compare
qdm12
force-pushed
the
qdm12/trie/subvalue-disk
branch
from
3261e9f to
539a755
Compare
qdm12
force-pushed
the
qdm12/trie/node-hashes-naming
branch
from
9a3c287 to
b00037d
Compare
- Record Merkle values deleted only if they were not changed since the last trie snapshot - Record Merkle values deleted when one or more nodes are deleted from the trie - Do NOT record Merkle values of inlined nodes in the deleted set of Merkle values (since they are not stored to database) - Lazy calculate Merkle values if they are missing (especially for delete operations) - Update tests to have full trie.go coverage again - Fix preparing branch for mutation only once - Fix merkle value calculation for future-to-be root node
- Remove `prepLeafForMutation` - Remove `prepBranchForMutation` - Remove `updateGeneration` - Replace above with `prepForMutation`
- Pruners only care about node hashes - Rename variables, functions and methods only dealing with node hashes - Do not write or read inlined nodes with a non-hash Merkle value - Clarify error wrappings and comments
qdm12
force-pushed
the
qdm12/trie/node-hashes-naming
branch
from
b00037d to
750bfcc
Compare
qdm12
force-pushed
the
qdm12/trie/subvalue-disk
branch
from
539a755 to
f4889ee
Compare
qdm12
force-pushed
the
qdm12/trie/node-hashes-naming
branch
2 times, most recently
from
acf44f4 to
b4c051e
Compare
qdm12
force-pushed
the
qdm12/trie/node-hashes-naming
branch
from
b4c051e to
e3aaddf
Compare
|
This is potentially quite wrong, so closing this PR. When inserting a (key, value) in the trie, the subvalue of the node to be created (even if it already exists in the trie) should be computed, and, if it's larger than 32B, the node inserted should be one of the new node variants (such as branch with hash of subvalue), where its key is the original key from the (key, value) inserted, but its storage value is the hash of the subvalue computed previously. Another node must be inserted (as v0 trie node variants such as branch with value) with its full key being the hash of the subvalue computed previously, and the storage value being the actual subvalue bytes from previously. Note this second node is part of the trie and the merkle computation. Another catch is that writing a (key, value) to the trie, where the key already exists and the value is the same as the one already there, should replace that node with the mechanism written in the paragraph above. This allows to later fully upgrade a v0 state trie to a v1 state trie. Finally when getting a storage value from the trie given a certain key, if the node at the key has a node variant of leaf/branch containing a hashed subvalue, the storage value to return should be extracted from the storage value (the subvalue initially computed when inserting) of the other node looked up using the subvalue hash. |
Changes
Tests
Issues
Partially address #2838 and #1936
Primary Reviewer
@timwu20