SIMD-0163: Lift the CPI caller restriction

[Lichtso]

No description provided.

[buffalojoec]

This is a great change! Eliminating the requirement for programs to contain the callee program account - in the cases of loaders v2 and v4 - will be a drastic improvement on CUs.

[cavemanloverboy]

Can you clarify why this is a massive improvement in CU cost? It seems to me like it's more of a general improvement to VM performance and just removes the need to construct one or a few more AccountInfos. Programs don't ever copy the program data.

As an extreme example, a noop program

pub extern "C" fn entrypoint(input: *mut u8) -> u64 {
    0
}

would see no reduction in CUs from being invoked with one less 10 MB account

[Lichtso]

@cavemanloverboy For top-level instructions this is in deed irrelevant. But, not so for nested CPI.

In nested CPI the program runtime does copy the program data (except for those owned by loader-v3) of instruction accounts, and taxes the caller for it here: https://github.com/anza-xyz/agave/blob/77b4d131502e095d18f3df7caab8b6a1cf7c1887/programs/bpf_loader/src/syscalls/cpi.rs#L886

[jstarry]

I really like this SIMD but I'm wondering if we should also update the tx format to make it explicit which programs may be invoked during tx execution. Rather than just appending program ids to the readonly accounts in a transaction, we could have a new message header value which defines how many of the trailing readonly accounts are invokable and should therefore be loaded into the program cache before execution.

[Lichtso]

we could have a new message header value which defines how many of the trailing readonly accounts are invokable

Good idea, however outside of this SIMD. This is only about CPI, it does not affect the message accounts.

[cavemanloverboy]

I agree. New transaction format needs to be thought through in great detail alongside #172.

I will draft something up in the next few weeks.

[2501babe]

should we specify that non-instruction non-payer accounts must be read-only and enforce this in sanitize? or specify that writable non-instruction non-payer accounts are demoted to read-only?

it wouldnt really affect lock contention because the only reason to append a writable account is to be annoying, and they could be just as annoying by taking it as an instruction account, but it would mean we dont need to check if a non-instruction account was invoked to skip it during account saving

[Lichtso]

I don't think this SIMD is that much related to account loading. It needs all accounts containing programs to be loaded in the program cache, that's it. And that happens anyway, independent of writeability and later account saving.

[joncinque]

It looks like there's consensus here. If no one objects within the next week, I'll merge this on 25 November.