-
Notifications
You must be signed in to change notification settings - Fork 14.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implements JSON-string connection representation generator #35723
Conversation
6be5197
to
fc96a40
Compare
airflow/models/connection.py
Outdated
@@ -496,3 +525,10 @@ def from_json(cls, value, conn_id=None) -> Connection: | |||
except ValueError: | |||
raise ValueError(f"Expected integer value for `port`, but got {port!r} instead.") | |||
return Connection(conn_id=conn_id, **kwargs) | |||
|
|||
@property | |||
def json_repr(self) -> str: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure about this one. Should it be a property or a method
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
property is fine
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In first glance yes it is fine, however if some additional settings would required in the future like indent or resolutions for empty hostname and schema which parsed from the uri we could extend it by optional parameters.
In the other hand it is just a helper for create connection for environment variable backend or some cloud based
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Personally I would do something like as_json()
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"There are only two hard things in Computer Science: cache invalidation and naming things"
I do not have personal preference, so let me change it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"There are only two hard things in Computer Science: cache invalidation and naming things"
The second is also off-by-one mistakes.
@@ -498,7 +498,7 @@ def serialize( | |||
type_=DAT.SIMPLE_TASK_INSTANCE, | |||
) | |||
elif isinstance(var, Connection): | |||
return cls._encode(var.to_dict(), type_=DAT.CONNECTION) | |||
return cls._encode(var.to_json_dict(validate=True), type_=DAT.CONNECTION) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isn't that going to be backwards incompatible for older connections?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It should be backward compatible.
If connection serialized by previous version, it should be deserialized by new one. I've also add test for that check.
I'm also not sure that we ever serialize connection in Airflow.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Previous version serialize conn_id, description and uri new one every field except uri.
So this is basically two existed ways to create connection from constructor.
Co-authored-by: Vincent <[email protected]>
* Implements JSON-string connection representation generator * json_repr -> as_json() * Apply suggestions from code review Co-authored-by: Vincent <[email protected]> --------- Co-authored-by: Vincent <[email protected]>
Airflow 2.3.0 introduce new way for store connection JSON, however there is no easy way to:
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rst
or{issue_number}.significant.rst
, in newsfragments.