services/horizon/internal/db2/history: Use FastBatchInsertBuilder to insert operations into history_operations #4952
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR Checklist
PR Structure
otherwise).
services/friendbot
, orall
ordoc
if the changes are broad or impact manypackages.
Thoroughness
.md
files, etc... affected by this change). Take a look in the
docs
folder for a given service,like this one.
Release planning
needed with deprecations, added features, breaking changes, and DB schema changes.
semver, or if it's mainly a patch change. The PR is targeted at the next
release branch if it's not a patch change.
What
In #4916 we introduced a FastBatchInsertBuilder which can be used to quickly insert a large amount of rows into Postgres via the COPY command. In this PR, we use FastBatchInsertBuilder to insert operations into the history_operations table.
Why
As a part of #4908 , we want to update all the code which inserts data into the history tables to use the FastBatchInsertBuilder.
Known limitations
I had to upgrade postgres to version 10+ because the COPY commands were failing with
pq: invalid input syntax for type json
errors when running with postgres 9.6.see https://stackoverflow.com/questions/52334512/postgres-copy-with-command-strips-escapes-from-json-file-also-loads-each-line-a
We have discussed dropping support for postgres 9 and there seems to be a consensus that we should do this, see https://stellarfoundation.slack.com/archives/C02B04RMK/p1662747589237539