Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: duplicate logs #1397

Merged
merged 2 commits into from
Jan 5, 2025
Merged

fix: duplicate logs #1397

merged 2 commits into from
Jan 5, 2025

Conversation

kyscott18
Copy link
Collaborator

Description

The factory child address logic was not deduplicating its results, leading to syncLogsDynamic being called with many duplicate values. This function has some address batching logic, which would cause multiple replica eth_getLogs requests to be made. The resulting logs failed when being inserted into the database.

Example

  • factory contract with many duplicated child addresses

@kyscott18 kyscott18 linked an issue Jan 5, 2025 that may be closed by this pull request
@tk-o
Copy link

tk-o commented Jan 5, 2025

Awesome update, @kyscott18 🚀

@kyscott18 kyscott18 merged commit 787a8dc into main Jan 5, 2025
8 checks passed
@kyscott18 kyscott18 deleted the kjs/fix-duplicate-logs branch January 5, 2025 15:10
@github-actions github-actions bot mentioned this pull request Jan 5, 2025
@typedarray
Copy link
Collaborator

Nice find. To clarify - I see two changes here:

  1. When a user passes an array of hard-coded addresses in their config, de-duplicate the array while building the source/filter.
  2. When fetching child addresses from the sync store, use distinct to de-duplicate the result array (because its possible for a factory to emit multiple logs specifying the same child address).

The user bug reports were caused by the factory case during the historical sync - we would fetch an array of child addresses and include it in an eth_getLogs request. If that array contains duplicates, a compliant RPC should not return duplicate logs. But, we actually separate the child addresses into batches (50?) for eth_getLogs. So, we had a situation where the same address would be included in multiple batches, leading to duplicate logs even for compliant RPCs. Is that correct?

@kyscott18
Copy link
Collaborator Author

Correct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ON CONFLICT DO UPDATE command cannot affect row a second time
3 participants