Skip to content

Commit

Permalink
Remove check for at least one schema in GCSToBigquery (#18150)
Browse files Browse the repository at this point in the history
For the case when updating an existing table or insert data to a particular partition, no schema is needed.
Autodetect doesn't always work, e.g. cannot distinguish partition correctly. Other options requires forking the schema to airflow.
  • Loading branch information
c-nuro authored Sep 18, 2021
1 parent a54f8d6 commit 7458f1e
Showing 1 changed file with 0 additions and 5 deletions.
5 changes: 0 additions & 5 deletions airflow/providers/google/cloud/transfers/gcs_to_bigquery.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@
import json
from typing import Optional, Sequence, Union

from airflow.exceptions import AirflowException
from airflow.models import BaseOperator
from airflow.providers.google.cloud.hooks.bigquery import BigQueryHook
from airflow.providers.google.cloud.hooks.gcs import GCSHook
Expand Down Expand Up @@ -274,10 +273,6 @@ def execute(self, context):
object_name=self.schema_object,
)
schema_fields = json.loads(blob.decode("utf-8"))
elif self.schema_object is None and self.autodetect is False:
raise AirflowException(
'At least one of `schema_fields`, `schema_object`, or `autodetect` must be passed.'
)
else:
schema_fields = None

Expand Down

0 comments on commit 7458f1e

Please sign in to comment.