v0.8.5

Summary

This release is optional and contains minor fixes, updates to logging, and devops improvements.

• DataApi: Add endpoint on dataapi for ejection events
• EigenDAClient
  • Simplify confirmation depth and finalization logics for putting a blob
  • Refine error handling logics and error message if dispersal went wrong
  • Reuse a single underlying connection for all grpc methods
• Bug fix: Global rate limit mistrigger

What's Changed

• Fix printf lint warning by @estensen in #742
• bug: fix slice init length by @jingchanglu in #767
• Operator network scanning tool by @pschork in #705
• readme: Delete obsolete subdirectories by @cosmoknightt in #769
• style: remove redundant logs by returning msg in errors by @samlaf in #776
• Fix grpc size comments by @jianoaix in #751
• [blobstore] Filter out expired blobs from query by @ian-shim in #692
• Define separate fallback RPC argument to work around URL escaping issues by @pschork in #777
• Reader fragment by @cody-littley in #749
• Fix docker bake tagging by @pschork in #782
• 771 read traffic generator by @cody-littley in #666
• Remove minibatcher by @ian-shim in #783
• [v2] Disperser and Node service protobuf by @ian-shim in #784
• chore: fix two small typos by @cosmoknightt in #795
• fix: ExpresseionValues -> ExpressionValues typo by @hopeyen in #796
• [payments] numeric incremental updates, query with ordering and lim by @hopeyen in #788
• [v2] Node v2 gRPC Entrypoint by @ian-shim in #797
• adds subgraph indexing of ejection manager events by @pschork in #798
• Validate protos workflow by @cody-littley in #799
• Table store by @cody-littley in #774
• Add observability for operator stake distributions by @jianoaix in #808
• Perf: cache the operator state by @jianoaix in #809
• Add feature that can disable traffic generator metrics. by @cody-littley in #807
• Adds operator ejections dataapi endpoint by @pschork in #810
• [payments] meterer structs and helpers by @hopeyen in #789
• [v2] Disperser API Server Entrypoint by @ian-shim in #785
• Break out chainio and crypto by @mooselumph in #805
• Fix race condition when rebuilding proto files. by @cody-littley in #813
• Style: improve api clients comments by @samlaf in #780
• Added README files in v2 proto directories. by @cody-littley in #812
• Update v2 disperser protos by @ian-shim in #816
• Add .vscode in gitignore by @ian-shim in #825
• [payments] meterer core logic by @hopeyen in #790
• Add traffic generator 2 docker file. by @cody-littley in #817
• Update docker-publish-release to not trigger on pull requests by @pschork in #822
• Core/v2 by @mooselumph in #824
• ci: add eigenda-client testnet-integration test to integration-tests.yml workflow by @samlaf in #820
• Remove shadow write from disperser by @ian-shim in #830
• refactor: make disperser client reuse same grpc connection by @samlaf in #826
• refactor: api error functions naming convention by @samlaf in #831
• Replace math/rand with crypto/rand for generating challenge by @ian-shim in #829
• Update api error convention by @ian-shim in #834
• Refactor Transactor into Reader/Writer by @mooselumph in #832
• [payments] placeholder paid blob dispersal api by @hopeyen in #791
• Protobufs for the relay by @cody-littley in #803
• [v2] Blob Metadata Store by @ian-shim in #818
• [v2] Blob Header Authentication by @ian-shim in #836
• Add stake percentage + operator address to ejection reporting (dataapi + cli) by @pschork in #815
• Fixes for traffic generator docker file by @cody-littley in #840
• feat!: eigenda client confirmation depth by @samlaf in #821
• Fix pointer issue by @mooselumph in #839
• Normalize allowlist lookups to use lowercase non-checksummed authenticated address by @pschork in #838
• [v2] s3 Blob Store by @ian-shim in #841
• [payments] disperser server metering by @hopeyen in #792
• [v2] BlobMetadataStore: Blob Certificate Operations by @ian-shim in #844
• TTL + table store by @cody-littley in #811
• S3 relay interface by @cody-littley in #833
• feat: eigenda client returns 503 errors (for failover purpose) by @samlaf in #828
• Revert "S3 relay interface" by @cody-littley in #850
• Use serialized bytes for commitments proto by @ian-shim in #851
• fix(disperser-client): RetrieveBlob grpc max size regression bug by @samlaf in #849
• Unexport payment signer by @ian-shim in #847
• quorum threshold update by @0x0aa0 in #729
• onchain socket by @0x0aa0 in #804
• [v2] Implement DisperseBlob method by @ian-shim in #842
• Log semver scan operator id as hex by @pschork in #802
• fix: make integration test use two quorums by @ian-shim in #855
• Add per quorum semever stake percentage metrics by @pschork in #856
• Fix docker build args and restore semver info on operator node by @pschork in #857
• fix(api): failover error bugs by @samlaf in #858

New Contributors

• @jingchanglu made their first contribution in #767
• @cosmoknightt made their first contribution in #769

Full Changelog: v0.8.4...v0.8.5

[Holesky] v0.8.5-rc.0

Summary

This pre-release is optional and contains minor fixes, updates to logging, and devops improvements.

• DataApi: Add endpoint on dataapi for ejection events
• EigenDAClient
  • Simplify confirmation depth and finalization logics for putting a blob
  • Refine error handling logics and error message if dispersal went wrong
  • Reuse a single underlying connection for all grpc methods
• Bug fix: Global rate limit mistrigger

What's Changed

• Fix printf lint warning by @estensen in #742
• bug: fix slice init length by @jingchanglu in #767
• Operator network scanning tool by @pschork in #705
• readme: Delete obsolete subdirectories by @cosmoknightt in #769
• style: remove redundant logs by returning msg in errors by @samlaf in #776
• Fix grpc size comments by @jianoaix in #751
• [blobstore] Filter out expired blobs from query by @ian-shim in #692
• Define separate fallback RPC argument to work around URL escaping issues by @pschork in #777
• Reader fragment by @cody-littley in #749
• Fix docker bake tagging by @pschork in #782
• 771 read traffic generator by @cody-littley in #666
• Remove minibatcher by @ian-shim in #783
• [v2] Disperser and Node service protobuf by @ian-shim in #784
• chore: fix two small typos by @cosmoknightt in #795
• fix: ExpresseionValues -> ExpressionValues typo by @hopeyen in #796
• [payments] numeric incremental updates, query with ordering and lim by @hopeyen in #788
• [v2] Node v2 gRPC Entrypoint by @ian-shim in #797
• adds subgraph indexing of ejection manager events by @pschork in #798
• Validate protos workflow by @cody-littley in #799
• Table store by @cody-littley in #774
• Add observability for operator stake distributions by @jianoaix in #808
• Perf: cache the operator state by @jianoaix in #809
• Add feature that can disable traffic generator metrics. by @cody-littley in #807
• Adds operator ejections dataapi endpoint by @pschork in #810
• [payments] meterer structs and helpers by @hopeyen in #789
• [v2] Disperser API Server Entrypoint by @ian-shim in #785
• Break out chainio and crypto by @mooselumph in #805
• Fix race condition when rebuilding proto files. by @cody-littley in #813
• Style: improve api clients comments by @samlaf in #780
• Added README files in v2 proto directories. by @cody-littley in #812
• Update v2 disperser protos by @ian-shim in #816
• Add .vscode in gitignore by @ian-shim in #825
• [payments] meterer core logic by @hopeyen in #790
• Add traffic generator 2 docker file. by @cody-littley in #817
• Update docker-publish-release to not trigger on pull requests by @pschork in #822
• Core/v2 by @mooselumph in #824
• ci: add eigenda-client testnet-integration test to integration-tests.yml workflow by @samlaf in #820
• Remove shadow write from disperser by @ian-shim in #830
• refactor: make disperser client reuse same grpc connection by @samlaf in #826
• refactor: api error functions naming convention by @samlaf in #831
• Replace math/rand with crypto/rand for generating challenge by @ian-shim in #829
• Update api error convention by @ian-shim in #834
• Refactor Transactor into Reader/Writer by @mooselumph in #832
• [payments] placeholder paid blob dispersal api by @hopeyen in #791
• Protobufs for the relay by @cody-littley in #803
• [v2] Blob Metadata Store by @ian-shim in #818
• [v2] Blob Header Authentication by @ian-shim in #836
• Add stake percentage + operator address to ejection reporting (dataapi + cli) by @pschork in #815
• Fixes for traffic generator docker file by @cody-littley in #840
• feat!: eigenda client confirmation depth by @samlaf in #821
• Fix pointer issue by @mooselumph in #839
• Normalize allowlist lookups to use lowercase non-checksummed authenticated address by @pschork in #838
• [v2] s3 Blob Store by @ian-shim in #841
• [payments] disperser server metering by @hopeyen in #792
• [v2] BlobMetadataStore: Blob Certificate Operations by @ian-shim in #844
• TTL + table store by @cody-littley in #811
• S3 relay interface by @cody-littley in #833
• feat: eigenda client returns 503 errors (for failover purpose) by @samlaf in #828
• Revert "S3 relay interface" by @cody-littley in #850
• Use serialized bytes for commitments proto by @ian-shim in #851
• fix(disperser-client): RetrieveBlob grpc max size regression bug by @samlaf in #849
• Unexport payment signer by @ian-shim in #847
• quorum threshold update by @0x0aa0 in #729
• onchain socket by @0x0aa0 in #804
• [v2] Implement DisperseBlob method by @ian-shim in #842
• Log semver scan operator id as hex by @pschork in #802
• fix: make integration test use two quorums by @ian-shim in #855
• Add per quorum semever stake percentage metrics by @pschork in #856
• Fix docker build args and restore semver info on operator node by @pschork in #857
• fix(api): failover error bugs by @samlaf in #858

New Contributors

• @jingchanglu made their first contribution in #767
• @cosmoknightt made their first contribution in #769
• @hopeyen made their first contribution in #796

Full Changelog: v0.8.4...v0.8.5-rc.0

v0.8.4

Summary

This release is optional and contains minor fixes, updates to logging, and devops improvements.

• Latency observability into encoding and retrieval endpoint
• Reduce metrics cardinality
• EigenDA client:
  • Logging update
  • Ability to initialize without configuring signer
• Improvement in protobuf compilation & docker build pipeline

NOTE: The encoding optimization introduced in v0.8.0, which is not compatible with the previous versions, will be enabled on 10/13 11AM PDT. Please ensure that DA node is upgraded to v0.8.0+ by 10/11. This breaking change only affects DA nodes and does not affect rollups or any users posting blobs.

What's Changed

• chore: refactor to single multistage dockerfile with bake file by @samlaf in #754
• feat: dockerfile for deploying contracts by @samlaf in #755
• Lightnode docker by @cody-littley in #726
• Bump actions/upload-artifact action by @ian-shim in #758
• [ci] Generate unit test coverage on PR by @ian-shim in #759
• Add registry and repository variables to docker bake for devops builds by @pschork in #760
• Fix docker-bake issues with devops builds by @pschork in #761
• doc: remove duplicate word by @xiaobei0715 in #753
• DAS protobuffs by @cody-littley in #710
• chore: cleaner eigenda_client logs by @samlaf in #764
• Make the latency accounting more accurate for encoder by @jianoaix in #766
• Lightnode kv store by @cody-littley in #721
• Add headers for routing on encoding streamer by @dmanc in #765
• 'make protoc' now utilizes docker workflow by default. by @cody-littley in #748
• Reduce the operator latency metric cardinality by @jianoaix in #768
• Instrument the retrieval latency stages by @jianoaix in #771
• rm sig check by @bxue-l2 in #770
• Fix docker-publish-release docker bake by @pschork in #772

New Contributors

• @xiaobei0715 made their first contribution in #753

Full Changelog: v0.8.3...v0.8.4

[Holesky] v0.8.4-rc.0

Summary

This pre-release is optional and contains minor fixes, updates to logging, and devops improvements.

• Latency observability into encoding and retrieval endpoint
• Reduce metrics cardinality
• EigenDA client:
  • Logging update
  • Ability to initialize without configuring signer
• Improvement in protobuf compilation & docker build pipeline

NOTE: The encoding optimization introduced in v0.8.0, which is not compatible with the previous versions, will be enabled on 10/13 11AM PDT. Please ensure that DA node is upgraded to v0.8.0+ by 10/11. This breaking change only affects DA nodes and does not affect rollups or any users posting blobs.

What's Changed

• chore: refactor to single multistage dockerfile with bake file by @samlaf in #754
• feat: dockerfile for deploying contracts by @samlaf in #755
• Lightnode docker by @cody-littley in #726
• Bump actions/upload-artifact action by @ian-shim in #758
• [ci] Generate unit test coverage on PR by @ian-shim in #759
• Add registry and repository variables to docker bake for devops builds by @pschork in #760
• Fix docker-bake issues with devops builds by @pschork in #761
• doc: remove duplicate word by @xiaobei0715 in #753
• DAS protobuffs by @cody-littley in #710
• chore: cleaner eigenda_client logs by @samlaf in #764
• Make the latency accounting more accurate for encoder by @jianoaix in #766
• Lightnode kv store by @cody-littley in #721
• Add headers for routing on encoding streamer by @dmanc in #765
• 'make protoc' now utilizes docker workflow by default. by @cody-littley in #748
• Reduce the operator latency metric cardinality by @jianoaix in #768
• Instrument the retrieval latency stages by @jianoaix in #771
• rm sig check by @bxue-l2 in #770

New Contributors

• @xiaobei0715 made their first contribution in #753

Full Changelog: v0.8.3...v0.8.4-rc.0

v0.8.3

Summary

This release is optional for users and operators to upgrade.

Notable changes

• Performance improvement: Zero serialization of chunks after being encoded in the dispersal flow, eliminating 20s to minutes overhead on dispersal latency
• Performance observability: Added metrics for blob age, blob encoding latency and system throughput
• Robustness fixes:
  • Added blob state transition invariant checking to make sure blob state changes are correct
  • Added blob state recovery for handling failed blobs stuck in DISPERSING state

What's Changed

• Downlevel more encoding streamer error log by @ian-shim in #689
• Check if blobs are expired before returning by @mooselumph in #695
• Add log for chunk encoding format by @jianoaix in #696
• Add BlobMinibatchMapping in minibatch store by @ian-shim in #683
• Create blob minibatch mappings by @ian-shim in #687
• [BatchConfirmer] Apply state transitions for full batches by @ian-shim in #691
• [1/N][zero serialization] Create data structure for passing encode chunks by @jianoaix in #690
• Add an API to fetch blobs from a given batch header hash by @dmanc in #688
• Update README.md by @Ocheretovich in #694
• [node] AttestBatch endpoint by @ian-shim in #676
• Split blob writer code out of larger PR. by @cody-littley in #685
• [2/N][zero serialization] Make Batcher operate on chunks without ser/deser by @jianoaix in #700
• use insecure credential by @bxue-l2 in #711
• Simplify minibatch data model by @ian-shim in #707
• [node] Refactor blob expiration key by @ian-shim in #693
• Compress signatures in minibatch response by @ian-shim in #699
• [node] Make GetBlobHeader compatible with minibatches by @ian-shim in #698
• [1/N][GPU encoder] Add benchmarking code and refactor encoding module by @dmanc in #715
• Add GetBulkBlobMetadata method in BlobMetadataStore by @ian-shim in #703
• Perf observability: Add blob age at key stages in blob's lifecycle by @jianoaix in #719
• Perf observability: Encoder data throughput and blob size breakdown by @jianoaix in #716
• Perf observability: Add blob encoding latency at client side by @jianoaix in #722
• Fix stringification of uint8 by @jianoaix in #723
• [node] Make GetChunks compatible with minibatches by @ian-shim in #712
• Send blob headers to all quorums on minibatch dispersal by @ian-shim in #717
• [3/N][zero serialization] Make Encoder produce and pass chunks in desired format by @jianoaix in #735
• [4/N][zero serialization] Add fallback to the encoder client when it cannot tell chunk encoding format by @jianoaix in #738
• More fixes of quorum (uint8) stringification by @jianoaix in #731
• Perf observability: Add attestation and encoding throughput by @jianoaix in #725
• Created blob verifier worker. by @cody-littley in #708
• Add dispersing transistion check by @bxue-l2 in #741
• Recover failed blobs in encoding streamer by @ian-shim in #733
• Fix: Append metadata when constructing batch by @ian-shim in #746
• Minor updates to error logs by @ian-shim in #747
• Created dockerized workflow for building protobufs. by @cody-littley in #734
• chore(inabox): multiplex inabox logs to stdout to help debug easier by @samlaf in #752
• Expose public ip service option on node plugin by @pschork in #756

New Contributors

• @Ocheretovich made their first contribution in #694

Full Changelog: v0.8.2...v0.8.3

[Holesky] v0.8.3-rc.0

Summary

This pre-release is optional for users and operators to upgrade.

Notable changes

• Performance improvement: Zero serialization of chunks after being encoded in the dispersal flow, eliminating 20s to minutes overhead on dispersal latency
• Performance observability: Added metrics for blob age, blob encoding latency and system throughput
• Robustness fixes:
  • Added blob state transition invariant checking to make sure blob state changes are correct
  • Added blob state recovery for handling failed blobs stuck in DISPERSING state

What's Changed

• Downlevel more encoding streamer error log by @ian-shim in #689
• Check if blobs are expired before returning by @mooselumph in #695
• Add log for chunk encoding format by @jianoaix in #696
• Add BlobMinibatchMapping in minibatch store by @ian-shim in #683
• Create blob minibatch mappings by @ian-shim in #687
• [BatchConfirmer] Apply state transitions for full batches by @ian-shim in #691
• [1/N][zero serialization] Create data structure for passing encode chunks by @jianoaix in #690
• Add an API to fetch blobs from a given batch header hash by @dmanc in #688
• Update README.md by @Ocheretovich in #694
• [node] AttestBatch endpoint by @ian-shim in #676
• Split blob writer code out of larger PR. by @cody-littley in #685
• [2/N][zero serialization] Make Batcher operate on chunks without ser/deser by @jianoaix in #700
• use insecure credential by @bxue-l2 in #711
• Simplify minibatch data model by @ian-shim in #707
• [node] Refactor blob expiration key by @ian-shim in #693
• Compress signatures in minibatch response by @ian-shim in #699
• [node] Make GetBlobHeader compatible with minibatches by @ian-shim in #698
• [1/N][GPU encoder] Add benchmarking code and refactor encoding module by @dmanc in #715
• Add GetBulkBlobMetadata method in BlobMetadataStore by @ian-shim in #703
• Perf observability: Add blob age at key stages in blob's lifecycle by @jianoaix in #719
• Perf observability: Encoder data throughput and blob size breakdown by @jianoaix in #716
• Perf observability: Add blob encoding latency at client side by @jianoaix in #722
• Fix stringification of uint8 by @jianoaix in #723
• [node] Make GetChunks compatible with minibatches by @ian-shim in #712
• Send blob headers to all quorums on minibatch dispersal by @ian-shim in #717
• [3/N][zero serialization] Make Encoder produce and pass chunks in desired format by @jianoaix in #735
• [4/N][zero serialization] Add fallback to the encoder client when it cannot tell chunk encoding format by @jianoaix in #738
• More fixes of quorum (uint8) stringification by @jianoaix in #731
• Perf observability: Add attestation and encoding throughput by @jianoaix in #725
• Created blob verifier worker. by @cody-littley in #708
• Add dispersing transistion check by @bxue-l2 in #741
• Recover failed blobs in encoding streamer by @ian-shim in #733
• Fix: Append metadata when constructing batch by @ian-shim in #746
• Minor updates to error logs by @ian-shim in #747
• Created dockerized workflow for building protobufs. by @cody-littley in #734

New Contributors

• @Ocheretovich made their first contribution in #694

Full Changelog: v0.8.2...v0.8.3-rc.0

v0.8.2

Summary

This is a patch release to fix a bug related to the blob state transition (#741). Without this fix, the blob may have wrong state transition from Confirmed back to Dispersing, which should never happen.

Users/rollups: No action needed. This is a system internal fix and should just work for users.

Operators: Not affected. No action needed.

Full Changelog: v0.8.1...v0.8.2

v0.8.1

Summary

This is a patch release to fix a bug related to gRPC dial option (#711) in dispersal client library in v0.8.0, so users can retrieve the 4MiB blobs correctly.

Users/rollups: if you are using dispersal client to retrieve 4MiB blobs, please upgrade to v0.8.1. No impact otherwise.

Operators: no impact, and no action is needed.

Full Changelog: v0.8.0...v0.8.1

v0.8.0

Summary

This release is optional for now, but strongly recommended.

In particular, it’ll become required for operators to upgrade to v0.8.0 or higher before 2024/09/20 on Mainnet.

Two changes are needed to complete the upgrade:

• Upgrade image to v0.8.0 or higher
• Set the node environment variable “NODE_SRS_LOAD=8388608”

Both are included in operator setup repo: https://github.com/Layr-Labs/eigenda-operator-setup/releases/tag/v0.8.0

Notable features

• EigenDA users - Max Blob size raised from 2MiB to 4MiB
  • Not enabled for custom quorums with less than 8 operators
• Operators - Gnark chunk encoding to reduce data size by about 30%
  • Not enabled for now
  • Scheduled to enable after operators upgrade to v0.8.0 or later version
• NodeInfo API at Node: Operators will start making the Node software version information available to the Disperser
• Churn approval expiration reduced from 60 mins to 15 mins: Churner can approve a churn request every 15 mins now

Other notable changes

• Fixed a bug in blob state transition in Batcher (reset DISPERSING blobs back to PROCESSING at Batcher start)
• Improved performance logging

Extra details for blob size limit and chunk encoding

Increasing max blob size is a notable feature enabled in the release. This feature allows EigenDA users to send blobs of 4MiB, previously 2MiB. For operators, they will need to update the NODE_SRS_LOAD to support larger blobs. The resource utilization on the node is expected to be almost identical under the same level of traffic.

This release also contains chunk encoding changes that are not enabled for now, but are scheduled to enable at some future release, to give operators ample time to upgrade. When enabled after the release post 2024/09/20, the nodes with versions before v0.8.0 will not be able to parse the chunks from Disperser.

We will make an announcement in that future release when the changes are enabled.

What's Changed

• Simplify the bytes decoding of chunks by @jianoaix in #607
• Pull ejector out of dataapi by @ian-shim in #608
• Update eigenlayer-middleware by @0x0aa0 in #611
• Record args in mock transactor by @ian-shim in #616
• [1/N] Chunk encoding optimization: Support encoding format with gnark by @jianoaix in #617
• fix dataapi docker build by @ian-shim in #618
• Use variable rate interval for blob thoughput metrics based on time interval by @pschork in #621
• Add configuration setting for high-level EigenDA client for returning as soon as the blob has confirmed by @teddyknox in #620
• Log the batch validation latency breakdown by @jianoaix in #619
• [TxnManager] Make pk wallet compatible with broadcast check by @ian-shim in #626
• Log request serialization time by @jianoaix in #625
• [inabox] Set BATCHER_FINALIZATION_BLOCK_DELAY to 0 by @ian-shim in #631
• [4/N] Chunk encoding optimization: Add bundle encoding by @jianoaix in #632
• Adds NodeInfo endpoint to retriever and dispersal services for observability by @pschork in #630
• Refactor cpu prover by @bxue-l2 in #629
• [2/N] Chunk encoding optimization: bench mark the gnark based encoding by @jianoaix in #624
• [3/N] Chunk encoding optimization: Add support of new encoding format… by @jianoaix in #628
• [minibatch] Add feature flag on batcher by @ian-shim in #634
• Ignore project files for the Jetbrains GoLand IDE. by @cody-littley in #639
• Fix fft_fr.IsPowerOfTwo(). (#83) by @cody-littley in #638
• Remove ejector from dataapi by @ian-shim in #633
• [5/N] Chunk encoding optimization: Add support of new encoding at Node by @jianoaix in #636
• Refactor dispatcher timeout by @ian-shim in #641
• Collect Node Info during reachability check if port scan was successful by @pschork in #640
• Add a metric for DB writing throughput by @jianoaix in #644
• Inabox readme improvements by @cody-littley in #643
• Revert "Refactor dispatcher timeout" by @ian-shim in #645
• Minibatch store interface by @ian-shim in #635
• Update node interface to enable minibatching by @ian-shim in #646
• [minibatch] Create minibatch from encoding streamer by @ian-shim in #637
• Dispatcher method to disperse blobs to node via StoreBlobs endpoint by @ian-shim in #648
• Generate inclusion proof with blob index instead of hash by @ian-shim in #649
• Clean up EnableDualQuorums flag by @ian-shim in #656
• Add Minibatcher by @ian-shim in #655
• Fix typos by @nnsW3 in #659
• Code hygiene: return error instead of bool by @jianoaix in #660
• Doc hygiene: api docs, repo readme, codebase structure by @jianoaix in #662
• [dispatcher] Add method to send AttestBatch request to nodes by @ian-shim in #664
• [minibatcher] Fullbatch state management by @ian-shim in #665
• Minibatch dynamodb client by @pschork in #652
• [6/N] Chunk encoding optimization: Disperser/Retriever support for new chunk encoding by @jianoaix in #650
• Average blob throughput over 9m window by @pschork in #667
• Enable shadow PutItem blob metadata requests to separate dynamo table for minibatch preprod testing by @pschork in #669
• Added metrics utility for generator. by @cody-littley in #673
• Added test utilities. by @cody-littley in #672
• add configurable max blob size by @bxue-l2 in #675
• Log blob size on request by @ian-shim in #678
• [batcher] Recover blob state by @ian-shim in #657
• add grpc recv buffer by @bxue-l2 in #679
• Configuration for new generator functionality. by @cody-littley in #674
• Downgrade encoder error logs by @ian-shim in #682
• [7/N] Chunk encoding optimization: Node support for mixed encoding by @jianoaix in #654
• Added blob table data structure. by @cody-littley in #677
• Keep batch state in Minibatcher by @ian-shim in #668
• Code hygiene: more properly name the chunk encoding format field by @jianoaix in #661
• Fix the operator stake share basis point calculation by @jianoaix in #658
• [node] Add StoreBlobs endpoint for minibatch streaming by @ian-shim in #671
• Add BatchConfirmer by @ian-shim in #670
• Add batch read operation in dynamodb client by @ian-shim in #684
• Adjust the churn approval interval by @jianoaix in #651
• Downlevel more encoding streamer error log by @ian-shim in #689
• Check if blobs are expired before returning by @mooselumph in #695
• Add log for chunk encoding format by @jianoaix in #696

New Contributors

• @nnsW3 made their first contribution in #659

Full Changelog: v0.7.4...v0.8.0

[Holesky] v0.8.0-rc.1

Summary

This pre-release is optional for now, but strongly recommended.

In particular, it’ll become required for operators to upgrade to v0.8.0 or higher before 2024/09/15 on testnet.

Two changes are needed to complete the upgrade:

• Upgrade image to v0.8.0 or higher
• Set the node environment variable “NODE_SRS_LOAD=8388608”

Both are included in release https://github.com/Layr-Labs/eigenda-operator-setup/releases/tag/v0.8.0-rc.1

Notable features

• EigenDA users - Max Blob size raised from 2MiB to 4MiB
  • Not enabled for custom quorums with less than 8 operators
• Operators - Gnark chunk encoding to reduce data size by about 30%
  • Not enabled for now
  • Scheduled to enable after operators upgrade to v0.8.0 or later version
• NodeInfo API at Node: Operators will start making the Node software version information available to the Disperser
• Churn approval expiration reduced from 60 mins to 15 mins: Churner can approve a churn request every 15 mins now

Other notable changes

• Fixed a bug in blob state transition in Batcher (reset DISPERSING blobs back to PROCESSING at Batcher start)
• Improved performance logging

Extra details for blob size limit and chunk encoding

Increasing max blob size is a notable feature enabled in the release. This feature allows EigenDA users to send blobs of 4MiB, previously 2MiB. For operators, they will need to update the NODE_SRS_LOAD to support larger blobs. The resource utilization on the node is expected to be almost identical under the same level of traffic.

This pre-release also contains chunk encoding changes that are not enabled for now, but are scheduled to enable at some future release, to give operators ample time to upgrade. When enabled after the release post 2024/09/15, the nodes with versions before v0.8.0 will not be able to parse the chunks from Disperser.

We will make an announcement in that future release when the changes are enabled.

What's Changed

• Simplify the bytes decoding of chunks by @jianoaix in #607
• Pull ejector out of dataapi by @ian-shim in #608
• Update eigenlayer-middleware by @0x0aa0 in #611
• Record args in mock transactor by @ian-shim in #616
• [1/N] Chunk encoding optimization: Support encoding format with gnark by @jianoaix in #617
• fix dataapi docker build by @ian-shim in #618
• Use variable rate interval for blob thoughput metrics based on time interval by @pschork in #621
• Add configuration setting for high-level EigenDA client for returning as soon as the blob has confirmed by @teddyknox in #620
• Log the batch validation latency breakdown by @jianoaix in #619
• [TxnManager] Make pk wallet compatible with broadcast check by @ian-shim in #626
• Log request serialization time by @jianoaix in #625
• [inabox] Set BATCHER_FINALIZATION_BLOCK_DELAY to 0 by @ian-shim in #631
• [4/N] Chunk encoding optimization: Add bundle encoding by @jianoaix in #632
• Adds NodeInfo endpoint to retriever and dispersal services for observability by @pschork in #630
• Refactor cpu prover by @bxue-l2 in #629
• [2/N] Chunk encoding optimization: bench mark the gnark based encoding by @jianoaix in #624
• [3/N] Chunk encoding optimization: Add support of new encoding format… by @jianoaix in #628
• [minibatch] Add feature flag on batcher by @ian-shim in #634
• Ignore project files for the Jetbrains GoLand IDE. by @cody-littley in #639
• Fix fft_fr.IsPowerOfTwo(). (#83) by @cody-littley in #638
• Remove ejector from dataapi by @ian-shim in #633
• [5/N] Chunk encoding optimization: Add support of new encoding at Node by @jianoaix in #636
• Refactor dispatcher timeout by @ian-shim in #641
• Collect Node Info during reachability check if port scan was successful by @pschork in #640
• Add a metric for DB writing throughput by @jianoaix in #644
• Inabox readme improvements by @cody-littley in #643
• Revert "Refactor dispatcher timeout" by @ian-shim in #645
• Minibatch store interface by @ian-shim in #635
• Update node interface to enable minibatching by @ian-shim in #646
• [minibatch] Create minibatch from encoding streamer by @ian-shim in #637
• Dispatcher method to disperse blobs to node via StoreBlobs endpoint by @ian-shim in #648
• Generate inclusion proof with blob index instead of hash by @ian-shim in #649
• Clean up EnableDualQuorums flag by @ian-shim in #656
• Add Minibatcher by @ian-shim in #655
• Fix typos by @nnsW3 in #659
• Code hygiene: return error instead of bool by @jianoaix in #660
• Doc hygiene: api docs, repo readme, codebase structure by @jianoaix in #662
• [dispatcher] Add method to send AttestBatch request to nodes by @ian-shim in #664
• [minibatcher] Fullbatch state management by @ian-shim in #665
• Minibatch dynamodb client by @pschork in #652
• [6/N] Chunk encoding optimization: Disperser/Retriever support for new chunk encoding by @jianoaix in #650
• Average blob throughput over 9m window by @pschork in #667
• Enable shadow PutItem blob metadata requests to separate dynamo table for minibatch preprod testing by @pschork in #669
• Added metrics utility for generator. by @cody-littley in #673
• Added test utilities. by @cody-littley in #672
• add configurable max blob size by @bxue-l2 in #675
• Log blob size on request by @ian-shim in #678
• [batcher] Recover blob state by @ian-shim in #657
• add grpc recv buffer by @bxue-l2 in #679
• Configuration for new generator functionality. by @cody-littley in #674
• Downgrade encoder error logs by @ian-shim in #682
• [7/N] Chunk encoding optimization: Node support for mixed encoding by @jianoaix in #654
• Added blob table data structure. by @cody-littley in #677
• Keep batch state in Minibatcher by @ian-shim in #668
• Code hygiene: more properly name the chunk encoding format field by @jianoaix in #661
• Fix the operator stake share basis point calculation by @jianoaix in #658
• [node] Add StoreBlobs endpoint for minibatch streaming by @ian-shim in #671
• Add BatchConfirmer by @ian-shim in #670
• Add batch read operation in dynamodb client by @ian-shim in #684
• Adjust the churn approval interval by @jianoaix in #651
• Downlevel more encoding streamer error log by @ian-shim in #689
• Check if blobs are expired before returning by @mooselumph in #695
• Add log for chunk encoding format by @jianoaix in #696

New Contributors

• @cody-littley made their first contribution in #639
• @nnsW3 made their first contribution in #659

Full Changelog: v0.7.4...v0.8.0-rc.1