Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

state: Add insert-only MPT implementation #477

Merged
merged 1 commit into from
Jun 29, 2022
Merged

state: Add insert-only MPT implementation #477

merged 1 commit into from
Jun 29, 2022

Conversation

chfast
Copy link
Member

@chfast chfast commented Jun 15, 2022

Add insert-only Merkle Patricia Trie implementation for computing MTP root hashes of (key, value) maps.

@codecov
Copy link

codecov bot commented Jun 15, 2022

Codecov Report

Merging #477 (af6425a) into master (2d3431d) will decrease coverage by 0.06%.
The diff coverage is 98.33%.

❗ Current head af6425a differs from pull request most recent head 9048c30. Consider uploading reports for the commit 9048c30 to get more accurate results

@@            Coverage Diff             @@
##           master     #477      +/-   ##
==========================================
- Coverage   99.61%   99.55%   -0.07%     
==========================================
  Files          42       44       +2     
  Lines        4702     4942     +240     
==========================================
+ Hits         4684     4920     +236     
- Misses         18       22       +4     
Flag Coverage Δ
consensus 79.12% <ø> (ø)
unittests 99.59% <98.33%> (-0.07%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
test/state/hash_utils.hpp 100.00% <ø> (ø)
test/state/mpt.cpp 96.82% <96.82%> (ø)
test/unittests/state_mpt_test.cpp 100.00% <100.00%> (ø)
test/utils/utils.hpp 100.00% <100.00%> (ø)

@gumb0
Copy link
Member

gumb0 commented Jun 15, 2022

typo in commit message: MTP => MPT

test/state/mpt.cpp Outdated Show resolved Hide resolved
test/state/mpt.cpp Outdated Show resolved Hide resolved
test/state/mpt.cpp Outdated Show resolved Hide resolved
test/state/mpt.cpp Outdated Show resolved Hide resolved

[[nodiscard]] Path head(size_t size) const noexcept
{
assert(size < length);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tail(0) can return entire original path, but head(length) cannot?

test/state/mpt.cpp Outdated Show resolved Hide resolved
test/state/mpt.cpp Outdated Show resolved Hide resolved
test/state/mpt.cpp Outdated Show resolved Hide resolved
test/state/mpt.cpp Outdated Show resolved Hide resolved
test/state/mpt.cpp Outdated Show resolved Hide resolved
else
n = std::move(children[0]);

MPTNode* branch = nullptr;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add some comment(s) what's happening in this part - converting to a branch etc...

test/state/mpt.cpp Outdated Show resolved Hide resolved
bytes m_value;
std::unique_ptr<MPTNode> children[num_children];

MPTNode(Kind kind, const Path& path, bytes&& value = {}) noexcept
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's better to have bytes value parameter (pass by value). As I remember, && is a universal reference, supposed to handle both r-value references and values, useful mostly in templates. Here you intend to always move into constructor.

Also in leaf() and insert().

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is universal reference only when combined with a template. Here this is rvalue reference. User is forced to move the bytes. This is enough for our use cases and eliminates unnecessary copies.

Passing by value is indeed more generic and provides better API.

test/state/mpt.cpp Outdated Show resolved Hide resolved
@chfast
Copy link
Member Author

chfast commented Jun 27, 2022

Inspired by the review, I changed the "insert" implementation in a way the nodes are built bottom-up. Therefore, they can be built completely in a functional way.

test/state/mpt.cpp Outdated Show resolved Hide resolved
new_idx,
std::make_unique<MPTNode>(leaf(path.tail(mismatch_pos + 1), std::move(value))));

*this = (mismatch_pos == 0) ?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if mismatch_pos == 1, it will create new extension node with 1-nibble path, is this correct?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Sometimes we need to extract common path prefix to "ext" node. If the paths differ at position 0 then there is not common prefix. I will document all these.

test/state/mpt.cpp Outdated Show resolved Hide resolved
test/state/mpt.cpp Outdated Show resolved Hide resolved

[[nodiscard]] Path head(size_t size) const noexcept
{
assert(size < length);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still not sure why this is not inclusive

Suggested change
assert(size < length);
assert(size <= length);

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The assert comes from how this is used in MPT: using head() or tail() it may produce empty path but never produces a full copy of a path. I added separate asserts for this scope narrowing.

test/unittests/state_mpt_test.cpp Outdated Show resolved Hide resolved
Add insert-only Merkle Patricia Trie implementation for computing MPT
root hashes of (key, value) maps.
@chfast chfast merged commit c3bb1cc into master Jun 29, 2022
@chfast chfast deleted the state_mpt branch June 29, 2022 19:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants