XComObjectStorageBackend
can't deserialize_value
on S3 path
#39797
Labels
area:core
kind:bug
This is a clearly a bug
needs-triage
label for new issues that we didn't triage yet
Apache Airflow version
2.9.1
If "Other Airflow 2 version" selected, which one?
No response
What happened?
I use s3 storage in xcom backend and it Error on xcom (
Invalid Key
), I find it because In_is_relative_to(ObjectStoragePath(data), p)
return False because p frombase_path
that containconn_id@...
butdata
from the xcom (str(base_path.joinpath(dag_id, run_id, task_id, f"{uuid.uuid4()}{suffix}"))
) that did't contain conn_id anymore. The_is_relative_to
checkstorage_options
(conn_id
) that make it return False.What you think should happen instead?
It should return True because
data
come fromp
(serialize_value and deserialize_value on XCOM).How to reproduce
Use
XComObjectStorageBackend
inxcombackend
. Andxcom_objectstorage_path = s3://conn_id@airflow-test/
withconn_id
=amazon web services, endpoint = "https://example.com"
.Operating System
docker airflow slim-2.9.1-python3.9 run on Ubuntu 22.04.4 LTS host.
Versions of Apache Airflow Providers
apache-airflow-providers-common-io 1.3.1
apache-airflow-providers-amazon 8.19.0
Deployment
Docker-Compose
Deployment details
No response
Anything else?
No response
Are you willing to submit PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: