GCSDeleteObjectsOperator raises unexpected ValueError for prefix set as empty string

[Lavedonio]

Apache Airflow Provider(s)

google

Versions of Apache Airflow Providers

All versions.

apache-airflow-providers-google>=1.0.0b1
apache-airflow-backport-providers-google>=2020.5.20rc1

Apache Airflow version

2.3.2 (latest released)

Operating System

macOS 12.3.1

Deployment

Composer

Deployment details

No response

What happened

I'm currently doing the upgrade check in Airflow 1.10.15 and one of the topics is to change the import locations from contrib to the specific provider.

While replacing:
airflow.contrib.operators.gcs_delete_operator.GoogleCloudStorageDeleteOperator
By:
airflow.providers.google.cloud.operators.gcs.GCSDeleteObjectsOperator

An error appeared in the UI: Broken DAG: [...] Either object or prefix should be set. Both are None

---

Upon further investigation, I found out that while the GoogleCloudStorageDeleteOperator from contrib module had this parameter check (as can be seen here):

assert objects is not None or prefix is not None

The new GCSDeleteObjectsOperator from Google provider module have the following (as can be seen here):

if not objects and not prefix:
    raise ValueError("Either object or prefix should be set. Both are None")

---

As it turns out, these conditions are not equivalent, because a variable prefix containing the value of an empty string won't raise an error on the first case, but will raise it in the second one.

What you think should happen instead

This behavior does not match with the documentation description, since using a prefix as an empty string is perfectly valid in case the user wants to delete all objects within the bucket.

Furthermore, there were no philosophical changes within the API in that timeframe. This code change happened in this commit, where the developer's intent was clearly to remove assertions, not to change the logic behind the validation. In fact, it even relates to a PR for this Airflow JIRA ticket.

How to reproduce

Add a GCSDeleteObjectsOperator with a parameter prefix="" to a DAG.

Example:

from datetime import datetime, timedelta

from airflow import DAG
from airflow.providers.google.cloud.operators.gcs import GCSDeleteObjectsOperator

with DAG('test_dag', schedule_interval=timedelta(days=1), start_date=datetime(2022, 1, 1)) as dag:
    task = GCSDeleteObjectsOperator(
        task_id='task_that_generates_ValueError',
        bucket_name='some_bucket',
        prefix=''
    )

Anything else

In my opinion, the error message wasn't very accurate as well, since it just breaks the DAG without pointing out which task is causing the issue. It took me 20 minutes to pinpoint the exact task in my case, since I was dealing with a DAG with a lot of tasks.

Adding the task_id to the error message could improve the developer experience in that case.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!