This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
gcs_to_bq: Support schema in different bucket #12520
Labels
good first issue
kind:feature
Feature Requests
provider:google
Google (including GCP) related issues
Description
Schema path (schema_object) for table in
GCSToBigQueryOperator
is downloading schema from source bucket.New feature should support support also pulling schemas from different bucket.
Use case / motivation
I guess there are lot of use-cases. To mention one concrete:
In case there is a schema change, we make 1 change (in terraform) and it works. Otherwise we would need to do terraform change, apply, then either update
schema_fields
in operator and release it or to allow adding fields on table load (which we don't necessarily want all the time).Related Issues
Not directly, but at least - #8280 - polish system tests to operator while at it
The text was updated successfully, but these errors were encountered: