Grid page visibility in the latest 2.3.1 release

[stepanof]

Apache Airflow version

2.3.1 (latest released)

What happened

Hello team!

I have faced the problem with Grid visibility in the latest 2.3.1 release.

My issue (example)

1. Here is the end of dag code:

...
<code>
...
telegram_alert = TelegramOperator(
    task_id='telegram_alert'
    ,telegram_conn_id='telegram_bot_id'
    ,chat_id='telegram_chat_id'
    ,text="blablabla"
    ,dag=dag
)
python_task_1 >> python_task_2 >> puller >> telegram_alert

Grid:

2. Delete task with telegram alert:
   Code in the end of dag code:

...
<code>
...
python_task_1 >> python_task_2 >> puller

Grid after deleting (Telegram alert task disappeared):

Grid after triggering dag again:

3. Return task with telegram alert in the dag as it was at the begining.
   Grid:
   

PS:

As I remember in 2.3.0 there wasn't such bug.
I tried to roll back airflow image (from 2.3.1 to 2.3.0) and restart containers but faced error during starting webserver:
alembic.util.exc.CommandError: Can't locate revision identified by '1de7bc13c950'

I see the same error when try to execute such command:
airflow db downgrade --to-version 2.3.0
Please help to me to roll back the airflow version or may be I can get some patch which fix this bug.

What you think should happen instead

The Grid must be visible at any time at any case

How to reproduce

Steps to reproduce:

1. Create dag with N tasks
2. Trigger dag
3. Delete 1 task from dag
4. Trigger dag
5. Return deleted task to dag
6. Try to see DagRun history on the Grid page.

Operating System

NAME="Ubuntu" VERSION="21.04 (Hirsute Hippo)" ID=ubuntu ID_LIKE=debian

Versions of Apache Airflow Providers

apache-airflow-providers-amazon==3.3.0
apache-airflow-providers-celery==2.1.4
apache-airflow-providers-docker==2.6.0
apache-airflow-providers-elasticsearch==3.0.3
apache-airflow-providers-google==6.8.0
apache-airflow-providers-grpc==2.0.4
apache-airflow-providers-hashicorp==2.2.0
apache-airflow-providers-http==2.1.2
apache-airflow-providers-imap==2.2.3
apache-airflow-providers-microsoft-azure==3.8.0
apache-airflow-providers-microsoft-mssql==2.1.3
apache-airflow-providers-mysql==2.2.3
apache-airflow-providers-odbc==2.0.4
apache-airflow-providers-postgres==4.1.0
apache-airflow-providers-redis==2.0.4
apache-airflow-providers-sendgrid==2.0.4
apache-airflow-providers-sftp==2.6.0
apache-airflow-providers-sqlite==2.1.3
apache-airflow-providers-ssh==2.4.3
apache-airflow-providers-telegram==2.0.4
apache-airflow-providers-vertica==2.1.3

Deployment

Docker-Compose

Deployment details

No response

Anything else

Many dags became unvisible on the Grid page and it is critical for us

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[stepanof]

WebServer log contains:

[2022-05-27 15:57:00,679] {manager.py:511} ERROR - Creation of Permission View Error: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "ab_permission_view_permission_id_view_menu_id_key"
DETAIL:  Key (permission_id, view_menu_id)=(13, 479) already exists.
[SQL: INSERT INTO ab_permission_view (id, permission_id, view_menu_id) VALUES (nextval('ab_permission_view_id_seq'), %(permission_id)s, %(view_menu_id)s) RETURNING ab_permission_view.id]
[parameters: {'permission_id': 13, 'view_menu_id': 479}]
(Background on this error at: http://sqlalche.me/e/14/gkpj)


[2022-05-27 15:57:00,795] {manager.py:570} ERROR - Add Permission to Role Error: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "ab_permission_view_role_permission_view_id_role_id_key"
DETAIL:  Key (permission_view_id, role_id)=(1648, 1) already exists.
[SQL: INSERT INTO ab_permission_view_role (id, permission_view_id, role_id) VALUES (nextval('ab_permission_view_role_id_seq'), %(permission_view_id)s, %(role_id)s) RETURNING ab_permission_view_role.id]
[parameters: {'permission_view_id': 1648, 'role_id': 1}]
(Background on this error at: http://sqlalche.me/e/14/gkpj)

[stepanof]

What DevTools found:

[stepanof]

Looks similar to problem in pull 23332
Similar things:
There is problem with visibility of grid actions as well I have.
There is problem with permissions as well my webserver log contain permissions error.

I can't go deeper. This is my limit of understanding.
I will be quite glad to your feedback and help.

[Smirn08]

@stepanof you can try to delete old dag dagruns at /dagrun/list/

[gbonazzoli]

@stepanof

Same problem here !

I have some DAGS that are not showing up in the grid layout.
In 2.3.0 there was no problem, it starts after upgrade Airflow to 2.3.1

I suspect that the cause is some problems on the permission due to problem with AUTH_LDAP config that I have.

[potiuk]

@gbonazzoli @stepanof almost for sure the problem is Duplicate of #23908 which is already solved in (very soon) 2.3.2 release where #23947 fixes it. It woudl be great if the both of you test the 2.3.2 RC1 that should appear likely within hours.

[stepanof]

@potiuk I will be happy to test 2.3.2 RC1. Please let me know how I can get it.

[gbonazzoli]

@potiuk @stepanof

I can snapshot my 2.3.1 Environment, install 2.3.2 RC1 and work on it in production.
And if there aren't important regressions keep it without regarding the final results on the grid page.

I only need that 2.3.2 RC1 is installable through pip install.

Is it possible ?

[potiuk]

I only need that 2.3.2 RC1 is installable through pip install.

Yep. Once we get it out you will be able to install it via pip. I will ping here when we do. It took a bit more time though.

[potiuk]

It's out https://pypi.org/project/apache-airflow/2.3.2rc1/

[stepanof]

@potiuk what about docker image? Are you going to publish 2.3.2rc1 to docker hub?

[potiuk]

@potiuk what about docker image? Are you going to publish 2.3.2rc1 to docker hub?

Just being pushed. 3.7 is out already.

[potiuk]

[potiuk]

All regular images ready. Now preparing slim images.

[stepanof]

Great, thanks a lot! I will be back soon with feedback about Grid visibility :)

[stepanof]

Everything works without errors.
Thanks a lot for your prompt help!!!

[gbonazzoli]

2.3.rc1 is good!

Everything works as far as the Grid Page is concerned!

It seems also faster !!!

I will let you know if we encounter some regression.

Thanks!

[hsnprsd]

Hi!
When will airflow 2.3.2 be released?

[potiuk]

https://lists.apache.org/thread/ry8vnxt4mptoos20cdm2wm856985poqw

[jedcunningham]

Closing as this is a dup of #23908 and is fixed by #23947. Thanks for the report and testing rc1 @stepanof & @gbonazzoli. I invite you both to test 2.3.2rc2 and vote on the release: https://lists.apache.org/thread/08x514oto8r7359qk44dqh7297270dd5

Btw, if you do ever want to downgrade in the future, you have to downgrade while still on 2.3.1, as that version has the migrations between 2.3.0 and 2.3.1, while 2.3.0 does not. So remember to downgrade the db first, then roll back to the old image.

[gbonazzoli]

@jedcunningham

I've just upgraded to 2.3.2rc2 and everything is ok.

We were running 2.3.2rc1 on production since last monday and we had non problem at all.

[potiuk]

Thanks for confirmiing @gbonazzoli :). Great to get such feedback from our users when release a bug fix !