Update Kubernetes library version and backport pickle-fix for Loggers #18797
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ashb
added
full tests needed
|
Maybe it is worth splitting this in two actually -- add the reducer for logging to core, but leave the Kube client library version unchanged? |
Yeah 2 PRs does make sense. |
|
Split the copyreg in core airflow in to #18798 which is fine to merge in for 2.2.0rc1 |
ashb
force-pushed
the
update-kube-client
branch
from
2503c53 to
30b37a8
Compare
kaxil
approved these changes
Oct 7, 2021
potiuk
approved these changes
Oct 7, 2021
jedcunningham
approved these changes
Oct 7, 2021
|
Merging now 2.2.0 is out. |
|
Wait -- making sure tests actually run first 😁 |
ashb
force-pushed
the
update-kube-client
branch
from
30b37a8 to
9eb6ebe
Compare
ashb
force-pushed
the
update-kube-client
branch
from
9eb6ebe to
c2028de
Compare
ashb
force-pushed
the
update-kube-client
branch
2 times, most recently
from
2c2c555 to
77661b7
Compare
|
Green enough. Merging to mainline for 2.3 -- we shouldn't pull this in to 2.2 series. |
|
Oh wait, the helm tests were cancelled. Those one's probably are important to run. 👀 |
uranusjr
requested changes
Nov 12, 2021
uranusjr
approved these changes
Jan 3, 2022
|
Hmmm helm tests are still timing out here, which means something is not right |
github-actions
bot
removed
the
stale
Previously we pinned this version as v12 as a change to Kube library internals meant v1.Pod objects now have a logger object inside them, and couldn't be pickled on Python 3.6. To fix that we have "backported" the change in Python 3.7 to make Logger objects be pickled "by name". (In Python 3.7 the change adds `__reduce__` methods on to the Logger and RootLogger objects, but here we achieve it `copyreg` stdlib module so we don't monkeypatch anything.) This fix is also applied in to airflow core in a separate commit, but we also apply it here in the provider so that cncf.kubernetes client library can be updated but still used with older versions of Airflow that don't have this fix in.
ashb
force-pushed
the
update-kube-client
branch
from
0fa2c58 to
0d1dc45
Compare
|
Better now after a rebase. |
4 tasks
|
Removing 2.3.0 milestone as this PR is in the provider release cycle. |
potiuk
pushed a commit
that referenced
this pull request
Mar 26, 2022
Previously we pinned this version as v12 as a change to Kube library internals meant v1.Pod objects now have a logger object inside them, and couldn't be pickled on Python 3.6. To fix that we have "backported" the change in Python 3.7 to make Logger objects be pickled "by name". (In Python 3.7 the change adds `__reduce__` methods on to the Logger and RootLogger objects, but here we achieve it `copyreg` stdlib module so we don't monkeypatch anything.) This fix is also applied in to airflow core in a separate commit, but we also apply it here in the provider so that cncf.kubernetes client library can be updated but still used with older versions of Airflow that don't have this fix in. (cherry picked from commit 7222f68)
ephraimbuddy
pushed a commit
that referenced
this pull request
Mar 26, 2022
Previously we pinned this version as v12 as a change to Kube library internals meant v1.Pod objects now have a logger object inside them, and couldn't be pickled on Python 3.6. To fix that we have "backported" the change in Python 3.7 to make Logger objects be pickled "by name". (In Python 3.7 the change adds `__reduce__` methods on to the Logger and RootLogger objects, but here we achieve it `copyreg` stdlib module so we don't monkeypatch anything.) This fix is also applied in to airflow core in a separate commit, but we also apply it here in the provider so that cncf.kubernetes client library can be updated but still used with older versions of Airflow that don't have this fix in. (cherry picked from commit 7222f68)
ephraimbuddy
added
type:misc/internal
ephraimbuddy
pushed a commit
that referenced
this pull request
Mar 26, 2022
Previously we pinned this version as v12 as a change to Kube library internals meant v1.Pod objects now have a logger object inside them, and couldn't be pickled on Python 3.6. To fix that we have "backported" the change in Python 3.7 to make Logger objects be pickled "by name". (In Python 3.7 the change adds `__reduce__` methods on to the Logger and RootLogger objects, but here we achieve it `copyreg` stdlib module so we don't monkeypatch anything.) This fix is also applied in to airflow core in a separate commit, but we also apply it here in the provider so that cncf.kubernetes client library can be updated but still used with older versions of Airflow that don't have this fix in. (cherry picked from commit 7222f68)
36 tasks
ephraimbuddy
pushed a commit
that referenced
this pull request
Mar 28, 2022
This reverts commit cb9cdf5.
ephraimbuddy
pushed a commit
that referenced
this pull request
Mar 28, 2022
This reverts commit cb9cdf5.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously we pinned this version as v12 as a change to Kube library
internals meant v1.Pod objects now have a logger object inside them, and
couldn't be pickled on Python 3.6.
To fix that we have "backported" the change in Python 3.7 to make Logger
objects be pickled "by name". (In Python 3.7 the change adds
__reduce__methods on to the Logger and RootLogger objects, but herewe achieve it
copyregstdlib module so we don't monkeypatchanything.)
This fix is also applied in to airflow core in a separate commit, but we
also apply it here in the provider so that cncf.kubernetes client
library can be updated but still used with older versions of Airflow
that don't have this fix in.
I think we should NOT merge this until after 2.2.0 RC is cut, as I don't think we have time to test the kube client upgrade doesn't break anything before the planned RC release.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.