Put Account's data in an Arc #15639
Put Account's data in an Arc #15639
Conversation
| /// lamports in the account | ||
| pub lamports: u64, | ||
| /// data held in this account | ||
| pub data: Arc<Vec<u8>>, |
There was a problem hiding this comment.
@sakridge , @carllin
I've been working on replay timing. We copy data around a lot. This is an attempt to greatly reduce copying data. I've prototyped this all out in a separate branch. Overall, I measure we're saving 30% of the time for a ledger verify of a recent mainnet-beta snapshot with a handful of changes that will each be discussed in their own pull request. This is one of the larger ones. It is intended to be consistent with comments in the code that we should return a ref or a cow from the cache. It looks like it needs to be an Arc vs Rc because we move accounts between threads frequently and Rc is no good. It looks like it needs to be an Arc vs Cow because of lifetime issues. I worked a change almost all the way through that attempted to add the right lifetimes to everything from the cache out and eventually ran into a few problems that looked fundamental. Perhaps this lifetime approach can work at some point.
Note this change doesn't 'work' all the way yet. In my prototype implementation, I made a lot of functions generic on the AnAccount trait. That allows Account and AccountNoData to coexist. It looks like they need to coexist because Account has to remain as is because it is an ABI? This is just a guess. If so, the internals of all our code would likely switch to generating and using AccountNoData instances, and we'd have to convert when we go to some api that requires an Account. This isn't different than cloning an Account today. This change is meant to give us a specific thing to discuss.
Some alternatives include the whole account being behind an Arc or the account being in an Arc AND the data being in an Arc. Updating the data and copying it gets quite expensive. Arc seems to give us the copy on write behavior I was imagining we'd want. Arc also frees us from the lifetime issues of Cow.
There was a problem hiding this comment.
@ryoqun Carl says you've worked on something similar.
There was a problem hiding this comment.
There was a problem hiding this comment.
@Lichtso and I have communicated some on this topic. I am happy to collaborate more. The lifetime and copies of accounts with data from account load and caching to locking, serialization, vms, program execution, deserialization, verification, rollbacks, temporary storage, permanent storage, etc. are all in a circle of life. Sounds fun!
mvines
added
CI
|
replaced by #15691 |
Problem
Summary of Changes
Fixes #