Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix behaviour of build/pull after recent Breeze changes #24657

Conversation

potiuk
Copy link
Member

@potiuk potiuk commented Jun 25, 2022

The behaviour of Breeze after some recent changes related to
pulling and building images in parallel have been slightly broken.
Nothing serious but slightly annoying behaviour:

  • when starting breeze shell, the image was attempted to be
    build even if it was not needed (but cache efficiency made it fast
    enough to not be too annoying (unless we updated to newer
    python base image

  • breeze pull command for "latest" branch makes no sense any more -
    we stopped pushing "latest" image to ghcr.io, we only push
    cache and "tagged" images. We are now turning --image-tag as
    required in "pull_image" and when someone specifies latest,
    error and helpful message is printed

  • --force-build flag in "shell-related-commands" was not
    properly propagated to build-image so it did not actually
    force image building.

All those problems are fixed now.


^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragement file, named {pr_number}.significant.rst, in newsfragments.

@potiuk potiuk force-pushed the fix-pull-build-behaviour-of-breeze-after-recent-changes branch 5 times, most recently from da30c3c to ab4b7e9 Compare June 25, 2022 17:02
@potiuk potiuk requested a review from dstandish June 25, 2022 17:18
@potiuk potiuk requested a review from kaxil as a code owner June 25, 2022 19:10
@potiuk
Copy link
Member Author

potiuk commented Jun 25, 2022

Nees #24659 to succeed.

@potiuk potiuk force-pushed the fix-pull-build-behaviour-of-breeze-after-recent-changes branch 2 times, most recently from 196c130 to 39b9be2 Compare June 26, 2022 09:32
The behaviour of Breeze after some recent changes related to
pulling and building images in parallel have been slightly broken.
Nothing serious but slightly annoying behaviour:

* when starting breeze shell, the image was attempted to be
  build even if it was not needed (but cache efficiency made it fast
  enough to not be too annoying (unless we updated to newer
  python base image

* breeze pull command for "latest" branch makes no sense any more -
  we stopped pushing "latest" image to ghcr.io, we only push
  cache and "tagged" images. We are now turning --image-tag as
  required in "pull_image" and when someone specifies latest,
  error and helpful message is printed

* --force-build flag in "shell-related-commands" was not
  properly propagated to build-image so it did not actually
  force image building.

All those problems are fixed now.
@potiuk potiuk force-pushed the fix-pull-build-behaviour-of-breeze-after-recent-changes branch from 39b9be2 to 90e0d15 Compare June 26, 2022 10:17
@potiuk
Copy link
Member Author

potiuk commented Jun 26, 2022

Would be nice to merge this one to get fixed non-rebuild experience

@github-actions
Copy link

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

@github-actions github-actions bot added the full tests needed We need to run full set of tests for this PR to merge label Jun 27, 2022
@potiuk potiuk merged commit b678dc2 into apache:main Jun 27, 2022
@potiuk potiuk deleted the fix-pull-build-behaviour-of-breeze-after-recent-changes branch June 27, 2022 01:00
potiuk added a commit to potiuk/airflow that referenced this pull request Jun 29, 2022
The behaviour of Breeze after some recent changes related to
pulling and building images in parallel have been slightly broken.
Nothing serious but slightly annoying behaviour:

* when starting breeze shell, the image was attempted to be
  build even if it was not needed (but cache efficiency made it fast
  enough to not be too annoying (unless we updated to newer
  python base image

* breeze pull command for "latest" branch makes no sense any more -
  we stopped pushing "latest" image to ghcr.io, we only push
  cache and "tagged" images. We are now turning --image-tag as
  required in "pull_image" and when someone specifies latest,
  error and helpful message is printed

* --force-build flag in "shell-related-commands" was not
  properly propagated to build-image so it did not actually
  force image building.

All those problems are fixed now.

(cherry picked from commit b678dc2)
@ephraimbuddy ephraimbuddy added this to the Airflow 2.3.3 milestone Jun 30, 2022
@ephraimbuddy ephraimbuddy added the changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) label Jul 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:dev-tools changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) full tests needed We need to run full set of tests for this PR to merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants