-
Notifications
You must be signed in to change notification settings - Fork 11.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
indexer fix: chunk to avoid PG parameter limit #19754
Conversation
The latest updates on your projects. Learn more about Vercel for Git ↗︎
3 Skipped Deployments
|
.execute(conn) | ||
.await?; | ||
for event_emit_packages_chunk in | ||
event_emit_packages.chunks(PG_COMMIT_CHUNK_SIZE_INTRA_DB_TX) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
anticipated q about PG_COMMIT_CHUNK_SIZE_INTRA_DB_TX
, we do not want to make this an env var / arg, b/c this is a protection to ensure commits can go thru, thus do not want operators to override, that might cause issues
## Description - fixed a bug that caused mainnet indexer to stop, also got reported in MystenLabs#19542, specifically here when tx has many input objects / affected objects / recipients / affected addresses etc. the expanded query will exceed the PG parameter limit of 65535 - also added ingestion tests for big tx indices & event indices and better error tracing ## Test plan added ingestion tests for tx and event indices --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API:
- fixed a bug that caused mainnet indexer to stop, also got reported in #19542, specifically here when tx has many input objects / affected objects / recipients / affected addresses etc. the expanded query will exceed the PG parameter limit of 65535 - also added ingestion tests for big tx indices & event indices and better error tracing added ingestion tests for tx and event indices --- Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API:
## Description cherry-pick pr ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API:
## Description - fixed a bug that caused mainnet indexer to stop, also got reported in #19542, specifically here when tx has many input objects / affected objects / recipients / affected addresses etc. the expanded query will exceed the PG parameter limit of 65535 - also added ingestion tests for big tx indices & event indices and better error tracing ## Test plan added ingestion tests for tx and event indices --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API:
Description
Test plan
added ingestion tests for tx and event indices
Release notes
Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.
For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates.