feat(contract-distribution): Filter out contract accesses that are newly-deployed and not present

[tayfunelmas]

Currently function call may fail if code does not exist (eg. code-hash of the account is either default-hash or wrong). In this case, the failure of executing the function comes later, so we already record that the missing contract is called.

To fix this, there are two approaches:

1. Identify that there is ContractNotPresent error when executing the function and not record the call if that happens.
2. When finalizing the ContractStorage, not include the code-hashes that a) are newly-deployed and 2) do not exist in the storage.

We take the second approach here, since the first approach led to non-clear code and prone to errors later as the code changes. The second approach introduces a storage read for the accesses, but we are already doing these reads in the current protocol (since we record the read of the contract code).

We update some unittests based on this change and add a new unittest for calling a missing contract (which fails without this change).

We also remove an optimization from the witness producer to exclude the deployments from accesses, since we are doing it in the ContractStorage now.

[tayfunelmas]

this is not needed anymore, since we are doing the filtering in the ContractStorage::finalize now.

[codecov]

Codecov Report

Attention: Patch coverage is 76.19048% with 5 lines in your changes missing coverage. Please review.

Project coverage is 71.19%. Comparing base (7a7f3ea) to head (7eeefec).
Report is 2 commits behind head on master.

Files with missing lines	Patch %	Lines
core/store/src/contract.rs	84.21%	1 Missing and 2 partials ⚠️
...src/stateless_validation/state_witness_producer.rs	0.00%	2 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #12350   +/-   ##
=======================================
  Coverage   71.19%   71.19%           
=======================================
  Files         839      839           
  Lines      169743   169755   +12     
  Branches   169743   169755   +12     
=======================================
+ Hits       120844   120854   +10     
+ Misses      43645    43643    -2     
- Partials     5254     5258    +4     

Flag	Coverage Δ
backward-compatibility	0.16% <0.00%> (-0.01%)	⬇️
db-migration	0.16% <0.00%> (-0.01%)	⬇️
genesis-check	1.22% <0.00%> (-0.01%)	⬇️
integration-tests	39.00% <76.19%> (+<0.01%)	⬆️
linux	70.65% <76.19%> (+<0.01%)	⬆️
linux-nightly	70.77% <76.19%> (-0.01%)	⬇️
macos	50.42% <61.90%> (+0.01%)	⬆️
pytests	1.53% <0.00%> (-0.01%)	⬇️
sanity-checks	1.34% <0.00%> (-0.01%)	⬇️
unittests	64.15% <76.19%> (+<0.01%)	⬆️
upgradability	0.21% <0.00%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[pugachAG]

doesn't this mean that we have to read all contract accesses from storage now? right now that happens rarely since most of them are served from the compiled contract cache

[tayfunelmas]

doesn't this mean that we have to read all contract accesses from storage now? right now that happens rarely since most of them are served from the compiled contract cache

Even though they are served from the compiled contract cache, we still read them from the storage here:

nearcore/core/store/src/trie/mod.rs

Line 1615 in 8e30ccd

let _ = self.get(&trie_key.to_vec());

because we need to include them (along with the internal nodes in trie) in the witness. let me know if I miss anything.

The compiled contract caches still optimizes the latency since we do not need to re-compiled the code.

The ideal setup would be some kind of contains method to check if key exists without returning the data but could not find it yet.

Another option is to not call record_call function if the executing the function returns code-not-present error (

nearcore/runtime/runtime/src/actions.rs

Line 122 in 8e30ccd

Err(VMRunnerError::ContractCodeNotPresent) => {

), which currently happens after the record_call call when executing the action (

nearcore/runtime/runtime/src/actions.rs

Line 216 in 8e30ccd

let outcome = execute_function_call(

), but I found that approach brittle.