Make a DAG for pointing ES aliases #3493
Labels
💻 aspect: code
Concerns the software code in the repository
🌟 goal: addition
Addition of new feature
🟨 priority: medium
Not blocking but should be addressed soon
🧱 stack: catalog
Related to the catalog and Airflow DAGs
Milestone
Problem
#2372 adds a DAG for creating ES indices, but does not have any way for promoting the index/applying an alias.
Description
For this issue, we should create a DAG that takes the following conf options:
source_index
: the name of the index that we want to apply an alias totarget_alias
: the name of the alias we want to applydelete_old_index
: a bool, default to False. Whether to delete the index previously pointed to by thetarget_alias
, if applicableIt should do the following:
target_alias
(it should skip if no such index exists)delete_old_index
is True, and we found an index in the first step, delete itWe should package this logic in a reusable TaskGroup, and also add it as an additional step to the
create_new_es_index
DAG from #2372.Alternatives
Alternatively, we could trigger this DAG from the
create_new_es_index
DAG. If possible it would be better to have those steps actually part of the index creation DAG, but simply reuse the code. This makes it much easier to follow when reading through a DagRun.The text was updated successfully, but these errors were encountered: