fix: generic trie value updates #12344
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #12344 +/- ##
===========================================
+ Coverage 38.11% 71.20% +33.09%
===========================================
Files 837 839 +2
Lines 168711 169781 +1070
Branches 168711 169781 +1070
===========================================
+ Hits 64298 120891 +56593
+ Misses 100525 43630 -56895
- Partials 3888 5260 +1372
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
core/primitives/src/state.rs
Outdated
| pub enum ValueToInsert { | ||
| /// Full value, works for all possible operations, triggers both memtrie | ||
| /// and disk changes generation. | ||
| Full(Vec<u8>), |
There was a problem hiding this comment.
We could store the hash to avoid double computations, but I like more this data layout.
There was a problem hiding this comment.
It would be nice to avoid double computation indeed! It can be significant (1us per hash)
There was a problem hiding this comment.
I want to fix it here, see #12344 (comment)
core/store/src/trie/mem/updating.rs
Outdated
| } | ||
| // We change refcount only when we also make disk updates. | ||
| // In this case, we must have Full value. | ||
| fn add_refcount_to_value(&mut self, value: ValueToInsert) { |
There was a problem hiding this comment.
It feels like the determination of whether to add refcount should be made based on a boolean that says "do I need disk updates", not whether the value to insert happens to be Full? I don't think the original interface was great either, but this kind of behavior feels implicit. Eh... but I guess I don't see a cleaner approach.
There was a problem hiding this comment.
Made naming more explicit: MemtrieAndDisk/MemtrieOnly
core/primitives/src/state.rs
Outdated
| pub enum ValueToInsert { | ||
| /// Full value, works for all possible operations, triggers both memtrie | ||
| /// and disk changes generation. | ||
| Full(Vec<u8>), |
There was a problem hiding this comment.
It would be nice to avoid double computation indeed! It can be significant (1us per hash)
Longarithm
force-pushed
the
trie-insert-interface
branch
from
526403a
to
cdf22dd
Compare
Longarithm
changed the title
Longarithm
force-pushed
the
trie-insert-interface
branch
from
9786d42
to
a273d14
Compare
|
@robin-near @Trisfald I'm sorry but I need additional round of review here, I realised something which changed my goal. I thought I didn't introduce additional hash computation, but I actually did. After looking at the whole flow, the easiest way to go forward seemed to be "do the right things now". Now, I additionally introduce As a nice consequence, for memtries hash computation is postponed after all inserts/deletes are processed. I think the confusion about hashes is partially caused by the fact that generally for memtries they are already known in Commit based on the previous version: a273d14 |
Preparation step for #12324.
Introduce
generic_store_valueandgeneric_delete_value. These are different ways for MemTrie and TrieStorage to process value updates. TLDR: memtries can work with inlined values; trie storages must always have full values. Also the way to record accessed nodes is different.Also, solving some issues on the way. For example, interfaces of
Trie::insertandMemTrieUpdate::insert_implcurrently diverge. The latter needs bothFlatStateValuefor memtrie changes andOption<Vec<u8>>for disk changes. There is a good reason for that - if memtrie is loaded from flat state, we don't need to produce disk trie changes, so it is enough to loadFlatStateValues.However, the current interface is quite loose, so it is a net improvement to make it more strict by requiring everyone to give
ValueUpdate::MemtrieAndDiskor::MemtrieOnly. The drawback is thatTrie::inserttechnically can receiveValueUpdate::MemtrieOnly, it just will panic. But well... it still looks better than unclear memtrie interface with two values and hashes on the way.