Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(Re)fix Dangling rows moving with MySQL+Replication #21061

Merged
merged 2 commits into from
Jan 24, 2022

Conversation

ashb
Copy link
Member

@ashb ashb commented Jan 24, 2022

Splitting this CREATE TABLE AS SELECT query into two queries (CREATE TABLE LIKE .. followed by INSERT INTO) because the former doesn't play nicely with MySQL when replication is enabled

This re-introduces the change from #19999 that I mistakenly removed in my refactor in #19808.

There aren't automatic tests for this as a) it's hard to set up, b) this code isn't touched very often anyway, and c) we don't have replicated MySQL in our tests either. So it's very unlikely this will happen again.


^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

Splitting this CREATE TABLE AS SELECT query into two queries (CREATE
TABLE LIKE .. followed by INSERT INTO) because the former doesn't play
nicely with MySQL when replication is enabled
@ashb ashb requested a review from uranusjr January 24, 2022 14:19
@ashb
Copy link
Member Author

ashb commented Jan 24, 2022

/cc @SamWheating re-introducing your change that I mistakenly removed. D'oh!

airflow/utils/db.py Outdated Show resolved Hide resolved
Co-authored-by: Jed Cunningham <[email protected]>
@github-actions
Copy link

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

@github-actions github-actions bot added the full tests needed We need to run full set of tests for this PR to merge label Jan 24, 2022
@github-actions
Copy link

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

Copy link
Contributor

@SamWheating SamWheating left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Looks good to me

@ashb ashb merged commit 07ea9fc into apache:main Jan 24, 2022
@ashb ashb deleted the mysql-replication-dangling-rows-refix branch January 24, 2022 20:37
@jedcunningham jedcunningham added the changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) label Mar 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) full tests needed We need to run full set of tests for this PR to merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants