Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: fix clean performance fix #530

Merged
merged 2 commits into from
Dec 2, 2021
Merged

fix: fix clean performance fix #530

merged 2 commits into from
Dec 2, 2021

Conversation

bethesque
Copy link
Member

@bethesque bethesque commented Dec 2, 2021

@barthez I worked out what the issue was. The joins needed to be fully qualified, because the second join .left_outer_join(:verifications, pact_version_id: :id) inferred that the join was meant to link the verifications table with the pact publications table, when it should have been the pact_versions table.

# broken query
db[:pact_versions]
.left_outer_join(:pact_publications, pact_version_id: :id)
.left_outer_join(:verifications, pact_version_id: :id)
.select(Sequel[:pact_versions][:id])
.where(
  Sequel[:pact_publications][:id] => nil,
  Sequel[:verifications][:id] => nil
=> #<Sequel::SQLite::Dataset: "SELECT `pact_versions`.`id` FROM `pact_versions` 
LEFT OUTER JOIN `pact_publications` ON (`pact_publications`.`pact_version_id` = `pact_versions`.`id`) 

# This is where it goes wrong 
LEFT OUTER JOIN `verifications` ON (`verifications`.`pact_version_id` = `pact_publications`.`id`) 

WHERE ((`pact_publications`.`id` IS NULL) AND (`verifications`.`id` IS NULL))">


# working query
db[:pact_versions]
.left_join(:pact_publications, Sequel[:pact_publications][:pact_version_id]=> Sequel[:pact_versions][:id])
.left_join(:verifications, Sequel[:verifications][:pact_version_id]=> Sequel[:pact_versions][:id])
.select(Sequel[:pact_versions][:id])
.where(
  Sequel[:pact_publications][:id] => nil,
  Sequel[:verifications][:id] => nil
=> #<Sequel::SQLite::Dataset: "SELECT `pact_versions`.`id` FROM `pact_versions` 
LEFT JOIN `pact_publications` ON (`pact_publications`.`pact_version_id` = `pact_versions`.`id`) 
LEFT JOIN `verifications` ON (`verifications`.`pact_version_id` = `pact_versions`.`id`) 
WHERE ((`pact_publications`.`id` IS NULL) AND (`verifications`.`id` IS NULL))">

barthez and others added 2 commits December 2, 2021 12:36
* fix(cleanup): Improve delete orphans SQL query

Original query to delete orphans was terribly slow and was unable to
finish within default 15s timeout. Instead of excluding all ids from
union, we left join all 3 tables and select only rows that have no
joined values in pact_publications or verifications.

Additionaly, the query was fixed for mysql. Whenever the Mysql error
occures instead of embedded query, ids are fetched and then directly
used in delete query.

* fix: Add sql_enable_caller_logging documentation

* fix: Fix unreliable spec
@bethesque bethesque merged commit 6c71e57 into master Dec 2, 2021
@bethesque bethesque deleted the feat/fix-clean-fix branch December 2, 2021 02:04
@barthez
Copy link
Contributor

barthez commented Dec 2, 2021

Oh! I see it now, thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants