SC Module Caching

[Eitu33]

Context

The SC execution recently moved from a system that compiles every bytecode before running it to a cache oriented system that stores compiled modules.

Here is its current state:

• Module caching first iteration massa-sc-runtime#207
• Module caching follow-up #3488

The purpose of this discussion is to expose the new behaviour and clarify the remaining grey areas.

Module Caching Flowchart

Initial proposal:

Second proposal (deterministic and with payments specified, relative changes are in blue)

Specifications

Data structures

• LRU cache: stands for Least Recently Used, stored in RAM and will contain the compiled modules of widely used smart contracts. When an SC is called, it moves up. The LRU has a fixed limit, if an SC falls under it will be relegated.
• HD cache: stands for Hard Drive, stored in disk memory and will contain the compiled modules (serialized version) of every smart contract in the ledger. It associates a reference counter to the modules.

Approximative gas prices:

• Add Bytecode if bytecode is not stored in the HD cache: Very high
• Add Bytecode if bytecode is stored in the HD cache: Very Low
• Delete Bytecode: Very Low
• Modify Bytecode: Similar to Add Bytecode
• Execute Bytecode: Low (if execution compilation is not taken into account)

Compilators

• Add Bytecode compilation: Cranelift (slow compilation but optimised output module)
• Execute Bytecode compilation: Ideal would be Singlepass (fast compilation but unoptimised output module) but because of relative gas costs will probably be Cranelift as well

Determinism

• LRU cache data is not necessarily deterministic
• HD cache data must be deterministic
• Cache system behaviour must be deterministic
• Ledger bytecode will be compiled and cached when bootstrapping

[Eitu33]

A question remains about the local execution of arbitrary bytecode. The initial solution was using Singlepass as the compiler to avoid making that kind of calls too heavy since they do not use cache most of the time. But doing so requires another set of gas costs specified for that particular case, the work this would require is not justified at the moment. So we are left with 2 choices here:

1. Keep the local execution of arbitrary bytecode available but use the Cranelift compiler and inform the user that this kind of practices are highly expensive in terms of gas
2. Remove or disable it

[damip]

• in the first and last flowcharts, compilation gas can or not be charged depending on the state of the LRU => non-determinism
• executeSC (deploys etc...), readonly, local executions etc... will all use cranelift => very heavy => let's keep singlepass for now and think of a double flowchart "cranelift for stored SCs, singlepass for the rest" as a followup

[Eitu33]

I agree with you a system using Cranelift and Singlepass seems like too much given the current state. And I omitted the ExecuteSC operation in #3560 (comment) so 2. is never an option.

[Eitu33]

in the first and last flowcharts, compilation gas can or not be charged depending on the state of the LRU => non-determinism

I pushed a second proposal.

[damip]

Extension to the CL/SP combination.

The LRU keys are in the format [SP/CL tag][hash].

Also when the CL-compiled version is available both in the LRU and the HD, we use the LRU module because it avoids deserializing.

In the HD, the keys are just [hash] and only the CL version is stored.

Memory caching should be enabled in the HD.

[Eitu33]

I agree with your flowchart but I have a question concerning the not in HD && in LRU as CL branches in add BC and exec BC. I don't understand this specification as I would have expected the not in HD && in LRU to always have a SP module. Also looking at your flowchart I see no branch where a CL module is added in the LRU without being added to the HD. Only present case that would create this result is delete BC on a BC that is contained in the LRU, as it would only delete the HD CL module. But do we really wish to handle the not in HD && in LRU as CL case for that reason only, or is there another one I'm unaware of? All in all I think it would be better if we stick with with a system where the only modules that can be present in the LRU without being in the HD are SP modules.

Memory caching should be enabled in the HD.

Also I'm not sure how you plan on handling that, I believe that is something that must be manually turned on by the user, it is not accessible by a program. It is also possible I didn't understand what you meant.

[damip]

"I would have expected the not in HD && in LRU to always have a SP module"

=> Example case: I add an SC on the chain, then delete it => it is not in the HD but is in LRU as CL

"I see no branch where a CL module is added in the LRU without being added to the HD"

=> the idea is that CL is better than SP, and on the HD we only want CL. When a CL is available, we add or upgrade any SP in the LRU towards the CL. Also getting pre-cooked CL modules from the LRU is faster than deserializing them from HD

[Eitu33]

For the record we agreed after a discussion that the behaviour to be implemented concerning this subject will be the one represented in the initial flowchart.

[Eitu33]

With non-deterministic and finite size HD cache:

Modules in blue would be handled in a threaded compilation manager. This is not required for the incoming implementation.

Edit:

Final implementation will be very similar to this with the following differences:

• no reference counter in HD
• random entry sniping when reaching max capacity in HD
• no LRU saving for EPH modules

[Eitu33]

Clear separation version

TLDR, here are the relative changes:

• no reference counter
• LRU treats only EPH
• HD treats only NEPH

[Eitu33]

Details update for the above scheme:

• EPH modules are not cached anymore
• LRU and HD cache NEPH modules only