Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: fix performance issues due to contention in the integrations table when publishing a large number of contracts (> 20) per request, in parallel #654

Merged

Conversation

bethesque
Copy link
Member

@bethesque bethesque commented Dec 13, 2023

For a particular customer who was publishing 19 contracts at a time with requests that overlapped, we were seeing that the lock on the integrations table was causing each subsequent request to take longer and longer as each waited for the lock on the integrations table to be released. The problem has been fixed by updating the integration rows all at once at the end of the request, rather than updating them one at a time when each pact gets saved.

Problematic way:

create/update pact 1
update integration 1
create/update pact 2
update integration 2
create/update pact 3
update integration 3
create/update pact 4
update integration 4

New way:

create/update pact 1
create/update pact 2
create/update pact 3
create/update pact 4
update integrations 1, 2, 3, 4

The code uses a pub/sub pattern using the Wisper library to update the contract_data_updated_at field on the integration when a consumer contract or a provider contract is published. The subscribed functions get invoked synchronously - the pub/sub is just a way to decouple the repositories so that the pacts/verifications repositories don't need to know about calling the integrations respository. It's possible that this layer of indirection is overkill 🤷🏽

The problem with this pattern is that it was causing the integration to be updated every time the pact was published, and because we now have an endpoint that allows the publication of multiple pacts at once, this was leading to the problem described above.

The new code only subscribes the integration listener to the individual resource endpoints (PUT pact, and POST verification), and manually batch updates the integrations in one go in the Contract Service, which was designed to handle the batch contract publications.

…le when publishing a large number of contracts (> 20) per request, in parallel
@bethesque bethesque requested a review from vwong December 13, 2023 05:56
@bethesque
Copy link
Member Author

cc @Saup21

Copy link
Contributor

@vwong vwong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice... This is where you can do a quick turn around, but not me!

@bethesque bethesque merged commit 321a229 into master Dec 14, 2023
17 checks passed
@bethesque bethesque deleted the PACT-1352-sequel-database-error-in-publish-contracts branch December 14, 2023 03:42
bethesque added a commit that referenced this pull request Mar 22, 2024
…le when publishing a large number of contracts (> 20) per request, in parallel (#654)

PACT-1352
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants