Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running airflow dags backfill --reset-dagruns <dag_id> -s <execution_start_dt> -e <execution_end_dt> results in error when run twice. #21058

Closed
2 tasks done
uplsh580 opened this issue Jan 24, 2022 · 4 comments · Fixed by #21116
Assignees
Labels
area:core kind:bug This is a clearly a bug

Comments

@uplsh580
Copy link
Contributor

Apache Airflow version

2.2.3 (latest released)

What happened

It's the same situation as #21023.
Only change to airflow dags backfill from airflow dags test.

(airflow) [www@np-data-eng-airflow-sync001-lde-jp2v-prod ~]$ airflow dags backfill tutorial --reset-dagruns -s 2022-01-20 -e 2022-01-23
You are about to delete these 9 tasks:
<TaskInstance: tutorial.print_date scheduled__2022-01-20T07:48:54.720148+00:00 [success]>
<TaskInstance: tutorial.print_date scheduled__2022-01-21T07:48:54.720148+00:00 [success]>
<TaskInstance: tutorial.print_date scheduled__2022-01-22T07:48:54.720148+00:00 [success]>
<TaskInstance: tutorial.sleep scheduled__2022-01-20T07:48:54.720148+00:00 [success]>
<TaskInstance: tutorial.sleep scheduled__2022-01-21T07:48:54.720148+00:00 [success]>
<TaskInstance: tutorial.sleep scheduled__2022-01-22T07:48:54.720148+00:00 [success]>
<TaskInstance: tutorial.templated scheduled__2022-01-20T07:48:54.720148+00:00 [success]>
<TaskInstance: tutorial.templated scheduled__2022-01-21T07:48:54.720148+00:00 [success]>
<TaskInstance: tutorial.templated scheduled__2022-01-22T07:48:54.720148+00:00 [success]>

Are you sure? (yes/no):
y
Traceback (most recent call last):
  File "/home1/www/venv3/airflow/bin/airflow", line 8, in <module>
    sys.exit(main())
  File "/home1/www/venv3/airflow/lib/python3.7/site-packages/airflow/__main__.py", line 48, in main
    args.func(args)
  File "/home1/www/venv3/airflow/lib/python3.7/site-packages/airflow/cli/cli_parser.py", line 48, in command
    return func(*args, **kwargs)
  File "/home1/www/venv3/airflow/lib/python3.7/site-packages/airflow/utils/cli.py", line 92, in wrapper
    return f(*args, **kwargs)
  File "/home1/www/venv3/airflow/lib/python3.7/site-packages/airflow/cli/commands/dag_command.py", line 108, in dag_backfill
    dag_run_state=State.NONE,
  File "/home1/www/venv3/airflow/lib/python3.7/site-packages/airflow/models/dag.py", line 1948, in clear_dags
    dry_run=False,
  File "/home1/www/venv3/airflow/lib/python3.7/site-packages/airflow/utils/session.py", line 70, in wrapper
    return func(*args, session=session, **kwargs)
  File "/home1/www/venv3/airflow/lib/python3.7/site-packages/airflow/models/dag.py", line 1887, in clear
    dag_run_state=dag_run_state,
  File "/home1/www/venv3/airflow/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 270, in clear_task_instances
    dr.state = dag_run_state
  File "<string>", line 1, in __set__
  File "/home1/www/venv3/airflow/lib/python3.7/site-packages/airflow/models/dagrun.py", line 194, in set_state
    raise ValueError(f"invalid DagRun state: {state}")
ValueError: invalid DagRun state: None

What you expected to happen

No response

How to reproduce

  1. Setup Airflow 2.2.3
  2. Run any dag (in my case, using tutorial dag file).
  3. Run again same dag by airflow dags backfill command.

Operating System

CentOS Linux 7.9

Versions of Apache Airflow Providers

Providers info
apache-airflow-providers-celery | 2.1.0
apache-airflow-providers-cncf-kubernetes | 3.0.1
apache-airflow-providers-ftp | 2.0.1
apache-airflow-providers-http | 2.0.2
apache-airflow-providers-imap | 2.1.0
apache-airflow-providers-mysql | 2.1.1
apache-airflow-providers-redis | 2.0.1
apache-airflow-providers-slack | 4.1.0
apache-airflow-providers-sqlite | 2.0.1
apache-airflow-providers-ssh | 2.3.0

Deployment

Virtualenv installation

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@uplsh580 uplsh580 added area:core kind:bug This is a clearly a bug labels Jan 24, 2022
@potiuk
Copy link
Member

potiuk commented Jan 24, 2022

Feel free to fix ! If you are fast, we can even cherry-pick to 2.2.4 :)

@uranusjr
Copy link
Member

We should also add a check in clear_task_instances to catch this mistake, otherwise future regressions are bound to happen.

@potiuk
Copy link
Member

potiuk commented Jan 24, 2022

We should also add a check in clear_task_instances to catch this mistake, otherwise future regressions are bound to happen.

Good point!

@uplsh580
Copy link
Contributor Author

uplsh580 commented Jan 27, 2022

I create the other PR because the documents(CONTRIBUTORS_QUICK_START.rst, TESTING.rst) also seemed to need an update.
#21140

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:core kind:bug This is a clearly a bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants