Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CPI Account Reuse #19762

Merged

Conversation

Lichtso
Copy link
Contributor

@Lichtso Lichtso commented Sep 10, 2021

Problem

CPI needs an overhaul for inter-operation between the two ABI versions (see #19191).

  • The code around CPI is hard to read & understand, needs cleanup
  • native_invoke performs two unnecessary account copy steps:
    • Rc::new(keyed_account.account.clone())
    • dst_keyed_account.try_account_ref_mut()?.set_data(src_keyed_account.data().to_vec());
  • Syscall translate_accounts creates new Account structs as well:
    • Rc::new(RefCell::new(AccountSharedData::from(Account { .. })))
  • Both syscall and native_invoke should be recycling the existing account structs instead of secretly having their own internally, in order to interoperate with ABI v2.

Summary of Changes

  • General code cleanup, deleted about 200 LOC net
  • Consolidated the code around callee_keyed_accounts, caller_write_privileges and program_indices of the CPI syscall and native_invoke both into one in InstructionProcessor::create_message().
  • Removed the two account copy steps from native_invoke.
  • Recycles the existing account structs in both syscall and native_invoke.

Fixes #

@Lichtso Lichtso added the CI Pull Request is ready to enter CI label Sep 10, 2021
@solana-grimes solana-grimes removed the CI Pull Request is ready to enter CI label Sep 10, 2021
@codecov
Copy link

codecov bot commented Sep 10, 2021

Codecov Report

Merging #19762 (81b3ba9) into master (4675fa6) will increase coverage by 0.1%.
The diff coverage is 83.3%.

@@            Coverage Diff            @@
##           master   #19762     +/-   ##
=========================================
+ Coverage    82.6%    82.8%   +0.1%     
=========================================
  Files         478      487      +9     
  Lines      133547   134483    +936     
=========================================
+ Hits       110369   111386   +1017     
+ Misses      23178    23097     -81     

@Lichtso Lichtso force-pushed the refactor/cpi_syscall_account_reuse branch 2 times, most recently from 691eb04 to 929c8b7 Compare September 14, 2021 10:45
@Lichtso Lichtso force-pushed the refactor/cpi_syscall_account_reuse branch from 929c8b7 to df1b07c Compare September 14, 2021 11:19
@Lichtso Lichtso force-pushed the refactor/cpi_syscall_account_reuse branch from f0c0aa8 to 8840341 Compare September 14, 2021 15:45
@Lichtso Lichtso force-pushed the refactor/cpi_syscall_account_reuse branch from 4a22aac to 78c468e Compare September 16, 2021 14:46
@Lichtso Lichtso force-pushed the refactor/cpi_syscall_account_reuse branch from 78c468e to b10f302 Compare September 16, 2021 15:04
@Lichtso Lichtso force-pushed the refactor/cpi_syscall_account_reuse branch from b10f302 to 898c628 Compare September 16, 2021 17:22
@Lichtso Lichtso force-pushed the refactor/cpi_syscall_account_reuse branch from 898c628 to 9357b39 Compare September 16, 2021 18:18
@Lichtso Lichtso requested a review from jackcmay September 16, 2021 18:31
@@ -247,9 +260,14 @@ pub fn serialize_parameters_aligned(
pub fn deserialize_parameters_aligned(
keyed_accounts: &[KeyedAccount],
buffer: &[u8],
account_lengths: &[usize],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is neccessary because a CPI call will change the length of the account within this call?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that was the missing puzzle piece I searched for the entire day.
The hidden purpose of the additional temporary account structs.

Because once the CPI calls recycle the existing account structs, it overwrites the length and so the deserialize of the caller program afterwards parses and writes back garbage.

let program_id_index = message.instructions[0].program_id_index as usize;

Ok((message, program_id, program_id_index))
Ok((message, caller_write_privileges, program_indices))
}

/// Entrypoint for a cross-program invocation from a native program
pub fn native_invoke(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was mainly a copy-paste from the BPF version, thanks for cleaning it up

@Lichtso Lichtso force-pushed the refactor/cpi_syscall_account_reuse branch from 3f01613 to 81b3ba9 Compare September 17, 2021 22:10
@Lichtso Lichtso merged commit 36f46e1 into solana-labs:master Sep 18, 2021
@Lichtso Lichtso deleted the refactor/cpi_syscall_account_reuse branch September 18, 2021 06:09
@Lichtso Lichtso mentioned this pull request Sep 30, 2021
dankelleher pushed a commit to identity-com/solana that referenced this pull request Nov 24, 2021
* Removes two account copy steps from InstructionProcessor::native_invoke().

* Moves gathering of keyed_accounts, caller_write_privileges and program_indices into InstructionProcessor::create_message().

* Explicitly routes the serialized account lengths to enable sharing of existing account structures.

* Recycles existing account structs in CPI syscall.
frits-metalogix added a commit to identity-com/solana that referenced this pull request Nov 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants