Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(vm): Fast VM integration #1949

Merged
merged 128 commits into from
Aug 9, 2024
Merged

feat(vm): Fast VM integration #1949

merged 128 commits into from
Aug 9, 2024

Conversation

joonazan
Copy link
Contributor

@joonazan joonazan commented May 15, 2024 •
edited by slowli
Loading

What ❔

Integrates the new VM into server codebase:

  • Allows to run the new VM in the state keeper / batch executor (but not the API server sandbox). The new VM is disabled by default everywhere and requires an explicit opt-in.
  • Introduces the new component, VM playground, to test the new VM.
  • Runs the load test with the new VM in CI.

Why ❔

New VM is several times more efficient as per preliminary benchmarks and is easier to maintain.

Checklist

  • PR title corresponds to the body of PR (we generate changelog entries from PRs).
  • Tests for the changes have been added / updated.
  • Documentation comments have been added / updated.
  • Code has been formatted via zk fmt and zk lint.

joonazan and others added 30 commits May 15, 2024 17:09
@joonazan
start integrating vm2
fab3811
@joonazan
rough version of account validation gas limit
7aadf9a
@joonazan
add BootloaderState
0846b25
@joonazan
putting TX in bootloader
26bcbcb
@joonazan
able to fail one test
2643d7e
@joonazan
properly implement history deletion
fa9e2fa
@joonazan
fix formatting
8f5ff12
@joonazan
pass first test
3022d1d
@joonazan
treat zero gas as bootloader ran out
b9eac2a
@joonazan
enable oracle test
d15348a
@joonazan
clippy fixes
3e211cd
@joonazan
enable more tests
5f7c227
@joonazan
enable more tests
e567d19
@joonazan
give bootloader free memory
84c2a7f
@joonazan
refactor hooks
d954782
@joonazan
add more logging; partially implement refunds
3b9100c
@joonazan
deduplicate bytecodes based on latest storage changes
420f9cf
@joonazan
implement TX fail/success correctly
b1f3297
@joonazan
implement state diffs for PubdataRequested
27538ac
@joonazan
implement event merging
6ddf8cc
@joonazan
integrate l2 to l1 messages
556bcbb
@joonazan
make send_and_transfer pass
3c7bd29
@joonazan
add pubdata
85594a3
@joonazan
report pubdata difference, not whole pubdata
533d92a
@montekki @joonazan
implements is_write_initial for world
59cefe3
@joonazan
implement storage slot cost on multivm side
b682773
@joonazan
report only the last execution's pubdata and logs
ca0f399
@joonazan
support storage logs in a super dirty way
202f956
@joonazan
now passes refund test
31ac603
@joonazan
enable the majority of tests
99ed680
slowli and others added 2 commits August 7, 2024 12:52
@slowli
Use vm2::FIRST_HEAP
0dfba61
@joonazan
Merge pull request #2598 from matter-labs/aov-pla-1009-merge-vm-featu…
e481efb 
…re-branch-into-main

chore(vm): Update VM revision and TODOs
@slowli slowli marked this pull request as ready for review August 8, 2024 06:08
@slowli slowli requested a review from a team as a code owner August 8, 2024 06:08
@slowli slowli changed the title feat: VM2 integration feat(vm): Fast VM integration Aug 8, 2024
slowli and others added 2 commits August 8, 2024 18:03
@slowli
Update from upstream
9a83049
@slowli
feat(vm): Allow running new VM in state keeper (#2616)
7da372f 
## What ❔

- Allows running the new VM in the state keeper.
- Adds a CI load test with the new VM.

## Why ❔

Allows to monitor new VM performance regressions.
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 9, 2024

Detected VM performance changes

Benchmark name change in estimated runtime change in number of opcodes executed
call_far -44.5% -320208 (-100.0%)
decode_shl_sub -92.1% -4748033 (-100.0%)
deploy_simple_contract +51.4% -16872 (-100.0%)
finish_eventful_frames +66.5% -786848 (-100.0%)
event_spam -60.3% -2670258 (-100.0%)
write_and_decode -90.9% -4748952 (-100.0%)
slot_hash_collision -91.2% -4748952 (-100.0%)
access_memory -89.2% -4747939 (-100.0%)

Changes in number of opcodes executed indicate that the gas price of the benchmark has changed, which causes it run out of gas at a different time. Or that it is behaving completely differently.

slowli
slowli approved these changes Aug 9, 2024
View reviewed changes
Copy link
Contributor

@slowli slowli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm obviously biased here, but this PR looks ready to merge for me. Risks are (probably) mitigated by making the new VM strictly opt-in.

yorik
yorik approved these changes Aug 9, 2024
View reviewed changes
Copy link
Contributor

@yorik yorik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve for CI

@slowli slowli added this pull request to the merge queue Aug 9, 2024
Merged via the queue into main with commit b752a54 Aug 9, 2024
55 checks passed
@slowli slowli deleted the jms-vm2 branch August 9, 2024 12:48
This was referenced Aug 9, 2024
Merged
Merged
github-merge-queue bot pushed a commit that referenced this pull request Aug 13, 2024
@zksync-era-bot
chore(main): release core 24.17.0 (#2625)
a0c7a03 
🤖 I have created a release *beep* *boop*
---


##
[24.17.0](core-v24.16.0...core-v24.17.0)
(2024-08-13)


### Features

* Allow tracking l2 fees for L2-based chains
([#2563](#2563))
([e3f7804](e3f7804))
* Remove old EN code
([#2595](#2595))
([8d31ebc](8d31ebc))
* **tee:** introduce get_tee_proofs RPC method for TEE proofs
([#2474](#2474))
([d40ff5f](d40ff5f))
* **vm:** Fast VM integration
([#1949](#1949))
([b752a54](b752a54))


### Bug Fixes

* query for prover API
([#2628](#2628))
([b8609eb](b8609eb))
* **vm:** Fix missing experimental VM config
([#2629](#2629))
([e07a39d](e07a39d))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

---------

Co-authored-by: zksync-era-bot <[email protected]>
github-merge-queue bot pushed a commit that referenced this pull request Aug 16, 2024
@zksync-era-bot
chore(main): release prover 16.4.0 (#2615)
51259b6 
🤖 I have created a release *beep* *boop*
---


##
[16.4.0](prover-v16.3.0...prover-v16.4.0)
(2024-08-16)


### Features

* Bump harness & gpu deps
([#2634](#2634))
([2a7d566](2a7d566))
* Poll the main node API for attestation status - relaxed (BFT-496)
([#2583](#2583))
([b45aa91](b45aa91))
* **vlog:** Report observability config, flush, and shutdown
([#2622](#2622))
([e23e661](e23e661))
* **vm:** Extract VM interface to separate crate
([#2638](#2638))
([cb9ac4e](cb9ac4e))
* **vm:** Fast VM integration
([#1949](#1949))
([b752a54](b752a54))


### Bug Fixes

* Bump prover dependencies & rust toolchain
([#2600](#2600))
([849c6a5](849c6a5))
* **prover:** Fix NWG
([#2590](#2590))
([9b58ae9](9b58ae9))
* **prover:** Updated README.md
([#2604](#2604))
([be9f357](be9f357))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@slowli slowli slowli approved these changes

@yorik yorik yorik approved these changes

@alexandrst88 alexandrst88 Awaiting requested review from alexandrst88 alexandrst88 is a code owner automatically assigned from matter-labs/devops

@artmakh artmakh Awaiting requested review from artmakh artmakh is a code owner automatically assigned from matter-labs/devops

@hatemosphere hatemosphere Awaiting requested review from hatemosphere hatemosphere is a code owner automatically assigned from matter-labs/devops

@onyxet onyxet Awaiting requested review from onyxet onyxet is a code owner automatically assigned from matter-labs/devops

@otani88 otani88 Awaiting requested review from otani88 otani88 is a code owner automatically assigned from matter-labs/devops

@iluwaa iluwaa Awaiting requested review from iluwaa iluwaa is a code owner automatically assigned from matter-labs/devops

@montekki montekki Awaiting requested review from montekki

@RomanBrodetski RomanBrodetski Awaiting requested review from RomanBrodetski

@popzxc popzxc Awaiting requested review from popzxc

@itegulov itegulov Awaiting requested review from itegulov

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@joonazan @yorik @slowli @montekki