Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(vlog): New vlog interface + opentelemtry improvements #2472

Merged
merged 7 commits into from
Jul 26, 2024
Merged

feat(vlog): New vlog interface + opentelemtry improvements #2472

merged 7 commits into from
Jul 26, 2024

Conversation

popzxc
Copy link
Member

@popzxc popzxc commented Jul 24, 2024

OK, that's a big PR for its purpose, sorry for that.

Essence of changes:

Nice observability installation from the config

We can now write the following:

    let observability_config = configs
        .observability
        .clone()
        .context("observability config")?;
    let _observability_guard = observability_config.install()?;

instead of having to manually map data from observability_config to ObservabilityBuilder methods.
This is done in a non-intrusive way: the functionality is under a feature flag in zksync_config which is only enabled for binaries.

Nicer ObservabilityBuilder interface

Previously we've had "everything in one place", now we have separate structures for logs, opentelemetry, and sentry. It would be nice to integrate prometheus there too, but it's a separate task.

More service information is exposed via opentelemetry

Previously only service name was exposed. Now we also expose pod name and namespace.
It's done in a non-intrusive way as well, where you can provide some of information if you have it, but otherwise it'll be populated from env variables (which is fine, because it's not a permanent configuration, but rather a machine descriptor).

@popzxc popzxc force-pushed the popzxc-otlp-improvements branch from db7597a to de91800 Compare July 24, 2024 11:21
itegulov
itegulov reviewed Jul 25, 2024
View reviewed changes
Copy link
Contributor

@itegulov itegulov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great refactor, just one nit from my side

core/bin/block_reverter/src/main.rs Show resolved Hide resolved
itegulov
itegulov previously approved these changes Jul 25, 2024
View reviewed changes
@popzxc popzxc enabled auto-merge July 26, 2024 07:53
@popzxc popzxc added this pull request to the merge queue Jul 26, 2024
@popzxc popzxc removed this pull request from the merge queue due to a manual request Jul 26, 2024
@popzxc popzxc added this pull request to the merge queue Jul 26, 2024
Merged via the queue into main with commit c0815cd Jul 26, 2024
55 checks passed
@popzxc popzxc deleted the popzxc-otlp-improvements branch July 26, 2024 08:29
This was referenced Jul 26, 2024
Merged
Merged
github-merge-queue bot pushed a commit that referenced this pull request Aug 1, 2024
@zksync-era-bot
chore(main): release core 24.13.0 (#2502)
4729435 
🤖 I have created a release *beep* *boop*
---


##
[24.13.0](core-v24.12.0...core-v24.13.0)
(2024-07-31)


### Features

* Add recovery tests to zk_supervisor
([#2444](#2444))
([0c0d10a](0c0d10a))
* Added a JSON RPC to simulating L1 for consensus attestation
([#2480](#2480))
([c6b3adf](c6b3adf))
* added dropping all attester certificates when doing hard fork
([#2529](#2529))
([5acd686](5acd686))
* **configs:** Do not panic if config is only partially filled
([#2545](#2545))
([db13fe3](db13fe3))
* **eth-sender:** Make eth-sender tests use blob txs + refactor of
eth-sender tests
([#2316](#2316))
([c8c8334](c8c8334))
* Introduce more tracing instrumentation
([#2523](#2523))
([79d407a](79d407a))
* Remove unused VKs, add docs for BWG
([#2468](#2468))
([2fa6bf0](2fa6bf0))
* Revisit base config values
([#2532](#2532))
([3fac8ac](3fac8ac))
* Server 10k gwei limit on gas price and 1M limit on pubdata price
([#2460](#2460))
([be238cc](be238cc))
* **vlog:** Implement otlp guard with force flush on drop
([#2536](#2536))
([c9f76e5](c9f76e5))
* **vlog:** New vlog interface + opentelemtry improvements
([#2472](#2472))
([c0815cd](c0815cd))
* **zk_toolbox:** add test upgrade subcommand to zk_toolbox
([#2515](#2515))
([1a12f5f](1a12f5f))
* **zk_toolbox:** use configs from the main repo
([#2470](#2470))
([4222d13](4222d13))


### Bug Fixes

* **contract verifier:** Fix config values
([#2510](#2510))
([3729468](3729468))
* fixed panic propagation
([#2525](#2525))
([e0fc58b](e0fc58b))
* **proof_data_handler:** Unlock jobs on transient errors
([#2486](#2486))
([7c336b1](7c336b1))
* **prover:** Parallelize circuit metadata uploading for BWG
([#2520](#2520))
([f49720f](f49720f))
* VM performance diff: don't show 0 as N/A
([#2276](#2276))
([2fa2249](2fa2249))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

---------

Co-authored-by: zksync-era-bot <[email protected]>
@zksync-era-bot zksync-era-bot mentioned this pull request Aug 2, 2024
Merged
github-merge-queue bot pushed a commit that referenced this pull request Aug 2, 2024
@zksync-era-bot
chore(main): release zk_toolbox 0.1.1 (#2582)
b2e1fe5 
🤖 I have created a release *beep* *boop*
---


##
[0.1.1](zk_toolbox-v0.1.0...zk_toolbox-v0.1.1)
(2024-08-02)


### Features

* Add recovery tests to zk_supervisor
([#2444](#2444))
([0c0d10a](0c0d10a))
* add revert tests (external node) to zk_toolbox
([#2408](#2408))
([3fbbee1](3fbbee1))
* add revert tests to zk_toolbox
([#2317](#2317))
([c9ad002](c9ad002))
* add zk toolbox
([#2005](#2005))
([60a633b](60a633b))
* Adding SLChainID
([#2547](#2547))
([656e830](656e830))
* Base Token Fundamentals
([#2204](#2204))
([39709f5](39709f5))
* change `zkSync` occurences to `ZKsync`
([#2227](#2227))
([0b4104d](0b4104d))
* **configs:** Do not panic if config is only partially filled
([#2545](#2545))
([db13fe3](db13fe3))
* **eth-watch:** Integrate decentralized upgrades
([#2401](#2401))
([5a48e10](5a48e10))
* L1 batch signing (BFT-474)
([#2414](#2414))
([ab699db](ab699db))
* Minimal External API Fetcher
([#2383](#2383))
([9f255c0](9f255c0))
* Poll the main node for the next batch to sign (BFT-496)
([#2544](#2544))
([22cf820](22cf820))
* Revisit base config values
([#2532](#2532))
([3fac8ac](3fac8ac))
* Support sending logs via OTLP
([#2556](#2556))
([1d206c0](1d206c0))
* Switch to using crates.io deps
([#2409](#2409))
([27fabaf](27fabaf))
* **toolbox:** add format and clippy to zk_toolbox ci
([#2100](#2100))
([49a5c3a](49a5c3a))
* **toolbox:** add verify to zk-toolbox
([#2013](#2013))
([23a545c](23a545c))
* **toolbox:** add zk supervisor database commands
([#2051](#2051))
([f99739b](f99739b))
* **toolbox:** add zk_toolbox ci
([#1985](#1985))
([4ab4922](4ab4922))
* **toolbox:** refactor config to its own crate
([#2063](#2063))
([5cfcc24](5cfcc24))
* Update to consensus 0.1.0-rc.4 (BFT-486)
([#2475](#2475))
([ff6b10c](ff6b10c))
* **vlog:** New vlog interface + opentelemtry improvements
([#2472](#2472))
([c0815cd](c0815cd))
* **zk toolbox:** External node support
([#2287](#2287))
([6384cad](6384cad))
* **zk_toolbox:** Add check for zksync repo path
([#2447](#2447))
([f1cbb74](f1cbb74))
* **zk_toolbox:** Add contract verifier support for zk toolbox
([#2420](#2420))
([d10a24b](d10a24b))
* **zk_toolbox:** Add grafana support
([#2557](#2557))
([f5aaefe](f5aaefe))
* **zk_toolbox:** Add prover generate-sk command
([#2222](#2222))
([40e0a95](40e0a95))
* **zk_toolbox:** Add prover init command
([#2298](#2298))
([159af3c](159af3c))
* **zk_toolbox:** Add prover run
([#2272](#2272))
([598ef7b](598ef7b))
* **zk_toolbox:** add test upgrade subcommand to zk_toolbox
([#2515](#2515))
([1a12f5f](1a12f5f))
* **zk_toolbox:** Add update command
([#2440](#2440))
([e2fa86f](e2fa86f))
* **zk_toolbox:** Allow toolbox find Zkstack.yaml in parent dirs
([#2430](#2430))
([0957119](0957119))
* **zk_toolbox:** Clean command
([#2387](#2387))
([52a4680](52a4680))
* **zk_toolbox:** Dev command
([#2347](#2347))
([f508ac1](f508ac1))
* **zk_toolbox:** Implement default upgrader deployment
([#2526](#2526))
([6d86959](6d86959))
* **zk_toolbox:** resume functionality
([#2376](#2376))
([e5e0473](e5e0473))
* **zk_toolbox:** Small adjustment for zk toolbox
([#2424](#2424))
([ce43c42](ce43c42))
* **zk_toolbox:** Update prover support
([#2533](#2533))
([63c92b6](63c92b6))
* **zk_toolbox:** Update reamde for toolbox
([#2531](#2531))
([d5ba7d8](d5ba7d8))
* **zk_toolbox:** use configs from the main repo
([#2470](#2470))
([4222d13](4222d13))
* **zk_toolbox:** Use docker compose instead of docker-compose
([#2195](#2195))
([2f528ec](2f528ec))
* **zk_toolbox:** use low level command for running verbose command"
([#2358](#2358))
([29671c8](29671c8))
* **zk-toolbox:** add balance check
([#2016](#2016))
([a8b8e4b](a8b8e4b))
* **zk-toolbox:** Deploy custom token
([#2329](#2329))
([3a8fed4](3a8fed4))


### Bug Fixes

* **api:** correct default fee data in eth call
([#2072](#2072))
([e71f6f9](e71f6f9))
* disable localhost wallets on external network interaction
([#2212](#2212))
([a00317d](a00317d))
* **house-keeper:** Fix queue size queries
([#2106](#2106))
([183502a](183502a))
* **toolbox:** Temporary disable fast mode for deploying l1 contracts …
([#2011](#2011))
([2a1d37b](2a1d37b))
* update rust toolchain version
([#2047](#2047))
([9fe5212](9fe5212))
* **zk_toolbox:** Add chain id for local wallet
([#2041](#2041))
([8e147c1](8e147c1))
* **zk_toolbox:** Fix error with balances
([#2034](#2034))
([5d23a3e](5d23a3e))
* **zk_toolbox:** Fix installation guide
([#2035](#2035))
([e9038be](e9038be))
* **zk_toolbox:** Fix protocol version
([#2118](#2118))
([67f6080](67f6080))
* **zk_toolbox:** improve readme to include containers command and cd
([#2073](#2073))
([5e5628f](5e5628f))
* **zk_toolbox:** Move l1 rpc to init stage
([#2074](#2074))
([c127ff1](c127ff1))
* **zk_toolbox:** readme added dependencies section and cleaned up
([#2044](#2044))
([78244c7](78244c7))
* **zk_toolbox:** Set proper pubdata sending mode
([#2507](#2507))
([21fbd77](21fbd77))
* **zk_toolbox:** Show balance
([#2254](#2254))
([f1d9f03](f1d9f03))
* **zk_toolbox:** Some small nit
([#2023](#2023))
([4e96e32](4e96e32))
* **zk_toolbox:** Use both folders for loading contracts
([#2030](#2030))
([97c6d5c](97c6d5c))
* **zk_toolbox:** Use existing ecosystem
([#2534](#2534))
([99fd2bd](99fd2bd))
* **zk_toolbox:** Use slug crate instead of self written function
([#2309](#2309))
([a61f273](a61f273))
* **zk_toolbox:** Use the same l2 address for shared and erc20 bridge
([#2260](#2260))
([26f2010](26f2010))
* **zk_tool:** Change some texts
([#2027](#2027))
([a6232c5](a6232c5))
* zk-toolbox integration tests ci
([#2226](#2226))
([3f521ac](3f521ac))


### Reverts

* "feat: Poll the main node for the next batch to sign (BFT-496)"
([#2574](#2574))
([72d3be8](72d3be8))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
github-merge-queue bot pushed a commit that referenced this pull request Aug 2, 2024
@zksync-era-bot @Deniallugo
chore(main): release prover 16.2.0 (#2482)
e44bfff 
🤖 I have created a release *beep* *boop*
---


##
[16.2.0](prover-v16.1.0...prover-v16.2.0)
(2024-08-02)


### Features

* **configs:** Do not panic if config is only partially filled
([#2545](#2545))
([db13fe3](db13fe3))
* Introduce more tracing instrumentation
([#2523](#2523))
([79d407a](79d407a))
* New prover documentation
([#2466](#2466))
([1b61d07](1b61d07))
* optimize LWG and NWG
([#2512](#2512))
([0d00650](0d00650))
* Poll the main node for the next batch to sign (BFT-496)
([#2544](#2544))
([22cf820](22cf820))
* Remove CI and docker images for CPU prover & compressor
([#2540](#2540))
([0dda805](0dda805))
* Remove unused VKs, add docs for BWG
([#2468](#2468))
([2fa6bf0](2fa6bf0))
* Support sending logs via OTLP
([#2556](#2556))
([1d206c0](1d206c0))
* Update to consensus 0.1.0-rc.4 (BFT-486)
([#2475](#2475))
([ff6b10c](ff6b10c))
* **vlog:** New vlog interface + opentelemtry improvements
([#2472](#2472))
([c0815cd](c0815cd))
* **witness_vector_generator:** Make it possible to run multiple wvg
instances in one binary
([#2493](#2493))
([572ad40](572ad40))
* **zk_toolbox:** use configs from the main repo
([#2470](#2470))
([4222d13](4222d13))


### Bug Fixes

* **prover:** Parallelize circuit metadata uploading for BWG
([#2520](#2520))
([f49720f](f49720f))
* **prover:** Reduce database connection overhead for BWG
([#2543](#2543))
([c2439e9](c2439e9))
* **witness_generator:** Only spawn 1 prometheus exporter per witness
generator
([#2492](#2492))
([b9cb222](b9cb222))


### Reverts

* "feat: Poll the main node for the next batch to sign (BFT-496)"
([#2574](#2574))
([72d3be8](72d3be8))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

---------

Co-authored-by: Danil <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@EmilLuta EmilLuta EmilLuta approved these changes

@itegulov itegulov itegulov left review comments

@Deniallugo Deniallugo Awaiting requested review from Deniallugo

@perekopskiy perekopskiy Awaiting requested review from perekopskiy

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@popzxc @itegulov @EmilLuta