Skip to content
This repository has been archived by the owner on Jan 13, 2022. It is now read-only.

[Bug] DB cleaner DAG cannot clean when tags or metadata are defective #522

Closed
mathemancer opened this issue Oct 27, 2020 · 0 comments · Fixed by #523
Closed

[Bug] DB cleaner DAG cannot clean when tags or metadata are defective #522

mathemancer opened this issue Oct 27, 2020 · 0 comments · Fixed by #523

Comments

@mathemancer
Copy link
Contributor

Bug Description

The new DB cleaner DAG (at src/cc_catalog_airflow/dags/cleaner_workflow.py) implemented in #517 fails whenever:

  • tags are missing defective, or
  • metadata is missing or defective

Expected behavior

Whenever the above are true, the cleaner DAG should log the defect, then clean the rest of the row.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Cleaner workflow parallelism cc-archive/cccatalog
1 participant
@mathemancer