Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Compiler-v2][VM] add metadata check for script #14099

Merged
merged 2 commits into from
Jul 29, 2024

Conversation

rahxephon89
Copy link
Contributor

@rahxephon89 rahxephon89 commented Jul 23, 2024

Description

This PR adds a check in the VM when running bytecode generated by an unstable compiler or containing unstable language feature on mainnet

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Other (specify)

How Has This Been Tested?

Added an e2e move test

Key Areas to Review

  1. logic of checking in the VM;
  2. coverage in e2e move tests.

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

Copy link

trunk-io bot commented Jul 23, 2024

⏱️ 3h 59m total CI duration on this PR
Job Cumulative Duration Recent Runs
execution-performance / single-node-performance 22m 🟩
forge-compat-test / forge 20m 🟩
forge-framework-upgrade-test / forge 20m 🟩
forge-e2e-test / forge 17m 🟩
rust-move-tests 15m 🟩
rust-move-tests 14m 🟩
rust-move-unit-coverage 14m 🟩
rust-move-unit-coverage 12m 🟩
rust-move-unit-coverage 12m 🟥🟩
rust-move-tests 10m 🟩
rust-move-tests 9m 🟩
rust-cargo-deny 9m 🟩🟩🟩🟩🟩
rust-move-tests 9m
general-lints 9m 🟩🟩🟩🟩🟩
rust-move-unit-coverage 9m
rust-move-unit-coverage 8m 🟩
check-dynamic-deps 7m 🟩🟩🟩🟩🟩 (+1 more)
rust-doc-tests 6m 🟩
test-target-determinator 4m 🟩
execution-performance / test-target-determinator 4m 🟩
check 3m 🟩
semgrep/ci 3m 🟩🟩🟩🟩🟩 (+1 more)
file_change_determinator 1m 🟩🟩🟩🟩 (+1 more)
file_change_determinator 59s 🟩🟩🟩🟩🟩
permission-check 24s 🟩🟩🟩🟩🟩 (+1 more)
permission-check 22s 🟩🟩🟩🟩🟩 (+1 more)
permission-check 22s 🟩🟩🟩🟩🟩 (+1 more)
permission-check 16s 🟩🟩🟩🟩🟩 (+1 more)
file_change_determinator 13s 🟩
Backport PR 7s 🟥
permission-check 4s 🟩
permission-check 2s 🟩
determine-docker-build-metadata 1s 🟩

🚨 2 jobs on the last run were significantly faster/slower than expected

Job Duration vs 7d avg Delta
forge-framework-upgrade-test / forge 20m 13m +54%
forge-compat-test / forge 20m 15m +33%

settingsfeedbackdocs ⋅ learn more about trunk.io

aptos-move/aptos-vm/src/aptos_vm.rs Show resolved Hide resolved
.features()
.is_enabled(FeatureFlag::REJECT_UNSTABLE_BYTECODE_FOR_SCRIPT)
{
let script = match CompiledScript::deserialize_with_config(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we cache the deserialization of compiled scripts here by bundling it with verify_no_event_emission_in_script (this also deserializes the script) under a single function which does the deserialization and then runs verification passes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why else branch in the code below? I think having

// deserialize here
if feature_enabled {
  self.reject_unstable_bytecode_for_script(&script)?;
}
verifier::event_validation::verify_no_event_emission_in_compiled_script(&script)?;

is sufficient?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link

codecov bot commented Jul 23, 2024

Codecov Report

Attention: Patch coverage is 2.70270% with 36 lines in your changes missing coverage. Please review.

Project coverage is 59.1%. Comparing base (f2a427f) to head (f87d6b4).
Report is 4 commits behind head on main.

Files Patch % Lines
aptos-move/aptos-vm/src/aptos_vm.rs 0.0% 28 Missing ⚠️
aptos-move/framework/src/module_metadata.rs 0.0% 7 Missing ⚠️
...tos-move/aptos-vm/src/verifier/event_validation.rs 0.0% 1 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (f2a427f) and HEAD (f87d6b4). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (f2a427f) HEAD (f87d6b4)
2 1
Additional details and impacted files
@@             Coverage Diff             @@
##             main   #14099       +/-   ##
===========================================
- Coverage    70.7%    59.1%    -11.7%     
===========================================
  Files        2338      827     -1511     
  Lines      466716   201015   -265701     
===========================================
- Hits       330314   118830   -211484     
+ Misses     136402    82185    -54217     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@rahxephon89 rahxephon89 force-pushed the teng/add-metadata-check-for-script branch from 5d6e8af to 06aa2a8 Compare July 23, 2024 19:30
@rahxephon89 rahxephon89 marked this pull request as ready for review July 23, 2024 19:45
Copy link
Contributor

@georgemitenkov georgemitenkov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, only cone comment about if-else: I think you can just deserialise outside of if branch and then use a single if statement to make things nicer.

if metadata.unstable {
return Err(PartialVMError::new(StatusCode::UNSTABLE_BYTECODE_REJECTED)
.with_message("script marked unstable cannot be run on mainnet".to_string())
.finish(Location::Undefined));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: probably can use Location::Script

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@rahxephon89 rahxephon89 force-pushed the teng/add-metadata-check-for-script branch from 06aa2a8 to 48ccde9 Compare July 23, 2024 23:35
Copy link
Contributor

@ziaptos ziaptos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good

@rahxephon89 rahxephon89 force-pushed the teng/add-metadata-check-for-script branch from 48ccde9 to 3a0bb8e Compare July 29, 2024 17:50
@rahxephon89 rahxephon89 force-pushed the teng/add-metadata-check-for-script branch from 3a0bb8e to f87d6b4 Compare July 29, 2024 17:59
@rahxephon89 rahxephon89 enabled auto-merge (squash) July 29, 2024 17:59

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

✅ Forge suite realistic_env_max_load success on f87d6b4eaefda33e5fdb817fe8d6aaf5f7d4bbdf

two traffics test: inner traffic : committed: 9229.879735749722 txn/s, latency: 4376.610376928381 ms, (p50: 4200 ms, p90: 4800 ms, p99: 12000 ms), latency samples: 3509420
two traffics test : committed: 100.03751076489287 txn/s, latency: 2289.5215909090907 ms, (p50: 2100 ms, p90: 2400 ms, p99: 8100 ms), latency samples: 1760
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.251, avg: 0.230", "QsPosToProposal: max: 1.953, avg: 1.837", "ConsensusProposalToOrdered: max: 0.312, avg: 0.297", "ConsensusOrderedToCommit: max: 0.408, avg: 0.396", "ConsensusProposalToCommit: max: 0.706, avg: 0.693"]
Max round gap was 1 [limit 4] at version 1017650. Max no progress secs was 5.923208 [limit 15] at version 1017650.
Test Ok

Copy link
Contributor

✅ Forge suite framework_upgrade success on 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5 ==> f87d6b4eaefda33e5fdb817fe8d6aaf5f7d4bbdf

Compatibility test results for 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5 ==> f87d6b4eaefda33e5fdb817fe8d6aaf5f7d4bbdf (PR)
Upgrade the nodes to version: f87d6b4eaefda33e5fdb817fe8d6aaf5f7d4bbdf
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1265.645337595996 txn/s, submitted: 1268.207372692344 txn/s, failed submission: 2.5620350963481697 txn/s, expired: 2.5620350963481697 txn/s, latency: 2613.812983069562 ms, (p50: 1800 ms, p90: 4800 ms, p99: 9700 ms), latency samples: 108680
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1025.357400861774 txn/s, submitted: 1027.761265186045 txn/s, failed submission: 2.4038643242709963 txn/s, expired: 2.4038643242709963 txn/s, latency: 2889.1966005967606 ms, (p50: 2100 ms, p90: 5100 ms, p99: 10200 ms), latency samples: 93840
5. check swarm health
Compatibility test for 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5 ==> f87d6b4eaefda33e5fdb817fe8d6aaf5f7d4bbdf passed
Upgrade the remaining nodes to version: f87d6b4eaefda33e5fdb817fe8d6aaf5f7d4bbdf
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1272.015505374058 txn/s, submitted: 1273.852161425547 txn/s, failed submission: 1.8366560514889452 txn/s, expired: 1.8366560514889452 txn/s, latency: 3075.1456683168317 ms, (p50: 2400 ms, p90: 5400 ms, p99: 9000 ms), latency samples: 96960
Test Ok

Copy link
Contributor

✅ Forge suite compat success on 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5 ==> f87d6b4eaefda33e5fdb817fe8d6aaf5f7d4bbdf

Compatibility test results for 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5 ==> f87d6b4eaefda33e5fdb817fe8d6aaf5f7d4bbdf (PR)
1. Check liveness of validators at old version: 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5
compatibility::simple-validator-upgrade::liveness-check : committed: 6410.678495012214 txn/s, latency: 4342.450806861962 ms, (p50: 3400 ms, p90: 6200 ms, p99: 26800 ms), latency samples: 275140
2. Upgrading first Validator to new version: f87d6b4eaefda33e5fdb817fe8d6aaf5f7d4bbdf
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 7706.694884893845 txn/s, latency: 3433.2946282737257 ms, (p50: 3800 ms, p90: 4100 ms, p99: 4200 ms), latency samples: 142040
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 6715.101883034462 txn/s, latency: 4439.2509172795635 ms, (p50: 4100 ms, p90: 7500 ms, p99: 8200 ms), latency samples: 242020
3. Upgrading rest of first batch to new version: f87d6b4eaefda33e5fdb817fe8d6aaf5f7d4bbdf
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 7297.720823341179 txn/s, latency: 3744.207809721399 ms, (p50: 4200 ms, p90: 4600 ms, p99: 4700 ms), latency samples: 134960
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 6569.7598401712285 txn/s, latency: 4615.814899406332 ms, (p50: 4300 ms, p90: 7900 ms, p99: 8400 ms), latency samples: 242560
4. upgrading second batch to new version: f87d6b4eaefda33e5fdb817fe8d6aaf5f7d4bbdf
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 7747.953287483951 txn/s, latency: 2979.5028621908127 ms, (p50: 2700 ms, p90: 4500 ms, p99: 6500 ms), latency samples: 169800
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 7274.726980881865 txn/s, latency: 4316.364798099762 ms, (p50: 3000 ms, p90: 10400 ms, p99: 15700 ms), latency samples: 294700
5. check swarm health
Compatibility test for 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5 ==> f87d6b4eaefda33e5fdb817fe8d6aaf5f7d4bbdf passed
Test Ok

@rahxephon89 rahxephon89 merged commit f991f0d into main Jul 29, 2024
80 checks passed
@rahxephon89 rahxephon89 deleted the teng/add-metadata-check-for-script branch July 29, 2024 18:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants