Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GCSFileTransformOperator: New templated fields source_object, destination_object #23328

Merged
merged 2 commits into from
Apr 28, 2022
Merged

GCSFileTransformOperator: New templated fields source_object, destination_object #23328

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
45019d3
`GCSFileTransformOperator`: New templated fields `source_object`, `de…
eladkal Apr 28, 2022
9be6c6e
make docstring more informative
eladkal Apr 28, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions airflow/providers/google/cloud/operators/gcs.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -536,8 +536,10 @@ class GCSFileTransformOperator(BaseOperator):
data from source, transform it and write the output to the local
destination file.

:param source_bucket: The key to be retrieved from GCS. (templated)
:param destination_bucket: The key to be written from GCS. (templated)
:param source_bucket: The bucket to locate the source_object. (templated)
:param source_object: The key to be retrieved from GCS. (templated)
:param destination_bucket: The bucket to upload the key after transformation. (templated)
:param destination_object: The key to be written in GCS. (templated)
eladkal marked this conversation as resolved.
Show resolved Hide resolved
:param transform_script: location of the executable transformation script or list of arguments
passed to subprocess ex. `['python', 'script.py', 10]`. (templated)
:param gcp_conn_id: The connection ID to use connecting to Google Cloud.
Expand All @@ -553,7 +555,9 @@ class GCSFileTransformOperator(BaseOperator):

template_fields: Sequence[str] = (
'source_bucket',
'source_object',
'destination_bucket',
'destination_object',
'transform_script',
'impersonation_chain',
)
Expand Down