Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

solana-program: improve VoteState::deserialize_into() #2146

Merged
merged 5 commits into from
Jul 19, 2024

Conversation

2501babe
Copy link
Member

Problem

earlier this year we implemented VoteState::deserialize_into(), a custom parser intended to be suitable for usage in a bpf context. we want to use the new parser everywhere because, with some optimizations, it is 7-20x faster than bincode depending on the input

HANA custom vs bincode
* we win: 10000, they win: 0
* our max 22.422µs, their max 471.89µs
* our avg 5.76µs, their avg 35.956µs
* avg speedup 30.195µs

it also uses half as much memory for V1_14_11 because conversion to Current happens during parsing

Summary of Changes

implement those optimizations, and also remove some constraints that make this parser behave differently from bincode::deserialize. this allows VoteState::deserialize_into to be used in optimized vote processing

in the future we would like to replace VoteState::deserialize to use deserialize_into for everything, but this requires testing on mainnet-beta because it affects sensitive parts of the codebase which cannot be feature-gated

part of solana-labs#35101

@2501babe 2501babe changed the title solana-program: imprive VoteState::deserialize_into() solana-program: improve VoteState::deserialize_into() Jul 16, 2024
@2501babe 2501babe marked this pull request as draft July 16, 2024 09:11
@2501babe 2501babe force-pushed the 20240716_votestate3_part1 branch 2 times, most recently from 6b33566 to c4c07e0 Compare July 16, 2024 10:02
@2501babe 2501babe marked this pull request as ready for review July 16, 2024 18:44
@2501babe 2501babe requested a review from alessandrod July 16, 2024 18:45
Copy link

@alessandrod alessandrod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! A couple of nits and see the CircBuf comment

sdk/program/src/vote/state/mod.rs Outdated Show resolved Hide resolved
sdk/program/src/vote/state/mod.rs Outdated Show resolved Hide resolved
@@ -86,21 +86,12 @@ fn read_prior_voters_into<T: AsRef<[u8]>>(
if !is_empty {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if is_empty=true and prior_voters.buf is filled, we diverge from bincode
i think? One could maliciously create such a vote state.

(I think our Arbitrary impl for CircBuf should probably additionally generate this)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its true that it would diverge. how could one create such a vote state tho? i wrote this based on the fact that it seemed there was no code path to do that, all changes to circbuf go through the insert function (hence also why the Arbitrary impl works that way)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m a hacker, I see this commit get deployed to mainnet, as soon as it is I craft a special payload to split the cluster

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i was gonna say some stuff about how such a payload cant be delivered because it would fail vote program ownership, but it turns out theres negligible perf benefit to the special logic anyway. so, onwards!

Copy link

@alessandrod alessandrod Jul 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the call site where I'm looking to use this (Bank::update_stakes_cache), I'm definitely seeing votes fail to deserialize, so surely this must be possible somehow?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure what itd be withput seeing an example. in any event, should be the same as bincode now

@2501babe 2501babe force-pushed the 20240716_votestate3_part1 branch from 1f39b48 to 8dab344 Compare July 18, 2024 14:25
@2501babe 2501babe force-pushed the 20240716_votestate3_part1 branch from 10a4593 to 2807669 Compare July 18, 2024 16:14
@2501babe 2501babe requested a review from alessandrod July 18, 2024 19:16
Copy link

@alessandrod alessandrod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great work!

@2501babe 2501babe merged commit 2c4b61e into anza-xyz:master Jul 19, 2024
51 checks passed
@2501babe 2501babe deleted the 20240716_votestate3_part1 branch July 19, 2024 15:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants