Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DagRun's start_date updated when user clears task of the running Dagrun #30124

Closed
2 tasks done
dimonchik-suvorov opened this issue Mar 15, 2023 · 0 comments · Fixed by #30125
Closed
2 tasks done

DagRun's start_date updated when user clears task of the running Dagrun #30124

dimonchik-suvorov opened this issue Mar 15, 2023 · 0 comments · Fixed by #30125
Labels
affected_version:2.5 Issues Reported for 2.5 area:core good first issue kind:bug This is a clearly a bug

Comments

@dimonchik-suvorov
Copy link
Contributor

dimonchik-suvorov commented Mar 15, 2023

Apache Airflow version

2.5.1

What happened

DagRun state and start_date are reset if somebody is clearing a task of the running DagRun.

What you think should happen instead

I think we should not reset DagRun state and start_date in it's in the running or queued states because it doesn't make any sense for me. state and start_date of the DgRun should remain the same in case somebody's clearing a task of the running DagRun

How to reproduce

Let's say we have a Dag with 2 tasks in it - short one and the long one:

dag = DAG(
    'dummy-dag',
    schedule_interval='@once',
    catchup=False,
)

DagContext.push_context_managed_dag(dag)

bash_success = BashOperator(
    task_id='bash-success',
    bash_command='echo "Start and finish"; exit 0',
    retries=0,
)

date_ind_success = BashOperator(
    task_id='bash-long-success',
    bash_command='echo "Start and finish"; sleep 300; exit 0',
)

Let's day we have a running Dagrun of this DAG. First task finishes in a second and the long one is still running. We have a start_date and duration set and the Dagrun is still running. It runs for example for a 30 secs (pic 1 and 2)

image

image

Then we are clearing the short task. It causes clear of the Dagrun state (to queued) and clears start_date like we have a new Dagrun (pic 3 and 4)
image
image

Operating System

Unix/MacOS

Versions of Apache Airflow Providers

No response

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@dimonchik-suvorov dimonchik-suvorov added area:core kind:bug This is a clearly a bug needs-triage label for new issues that we didn't triage yet labels Mar 15, 2023
@eladkal eladkal added good first issue affected_version:2.5 Issues Reported for 2.5 and removed needs-triage label for new issues that we didn't triage yet labels Apr 13, 2023
dimonchik-suvorov pushed a commit to dimonchik-suvorov/airflow that referenced this issue Apr 25, 2023
This PR is fixing the issue described in apache#30124. We should not reset DagRun's `state` and `start_date` in case somebody is clearing task of the running Dagrun
ephraimbuddy pushed a commit that referenced this issue May 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affected_version:2.5 Issues Reported for 2.5 area:core good first issue kind:bug This is a clearly a bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants