Feature: Allow set Dag Run resource into Dag Level permission

[joaopamaral]

This PR extends Dag's access_control feature to allow Dag Run resource permissions. For example, it can allow only some selected Dag with trigger permission for a specific role.

closes: #37329

Validation:

• Testing removing a can_create DAG Runs and can_delete DAGs from User role.

$ airflow roles del-perms User -a "can_create" -r "DAG Runs"
$ airflow roles del-perms User -a "can_delete" -r "DAGs"

• Including the can_create DAG Runs in dag tutorial and can_delete DAG in dag tutorial2.

with DAG(
    'tutorial',
    ...
    access_control={
        'User': {
            'DAG Runs': {'can_create'}
        }
    }
) as dag:

with DAG(
    'tutorial2',
    ....
    access_control={
        'User': {'can_delete'}  # keeping support to old style, it can be also {'DAGs': {'can_delete'}}
    }
) as dag:

• UI with disable buttons with delete and trigger function working when allowed:

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[joaopamaral]

There are some tests failing because there was an increase in the number of queries. I'll check it out this week.

[joaopamaral]

@shahar1, fyi I've included this change to skip the Dag Run permissions (from dag level) in admin role f182af9 . I think it will fix the db test issue.

[shahar1]

@shahar1, fyi I've included this change to skip the Dag Run permissions (from dag level) in admin role f182af9 . I think it will fix the db test issue.

Apologies for the late response - couple of comments before merging:

1. Thanks for fixing the DB test!
2. Could you please merge again from main so we could ensure all checks pass?
3. Only now I realized that there's no documentation nor newsfragment includede in this PR. I think that both are essential for other people being aware of this feature, and using it properly. Could you please update the docs. in docs/apache-airflow-providers-fab/auth-manager/access-control.rst and create newsframent/40703.feature.rst?

Please let me know when you're done, and I'll re-approve and hopefully merge :)

[joaopamaral]

Hey @shahar1, I've included the docs and I've dug a little more about the query count test and the difference is during the _sync_perm_for_dag:

Recorded query locations:
        dagbag.py:_sync_perm_for_dag:748 > override.py:sync_perm_for_dag:1111 > override.py:create_permission:1848 > override.py:get_permission:1819 > override.py:get_action:1688:     15
        dagbag.py:_sync_perm_for_dag:748 > override.py:sync_perm_for_dag:1111 > override.py:create_permission:1848 > override.py:get_permission:1820 > override.py:get_resource:1749:   15
        dagbag.py:_serialize_dag_capturing_errors:692 > dagbag.py:_sync_perm_for_dag:748 > override.py:sync_perm_for_dag:1111 > override.py:create_permission:1848 > override.py:get_permission:1823:   10


E           AssertionError: The expected number of db queries is 94 with extra margin: 10. The current number is 154.
E           Recorded query locations:
E               dagbag.py:_sync_perm_for_dag:748 > override.py:sync_perm_for_dag:1123 > override.py:create_permission:1890 > override.py:get_permission:1861 > override.py:get_action:1730:     35
E               dagbag.py:_sync_perm_for_dag:748 > override.py:sync_perm_for_dag:1123 > override.py:create_permission:1890 > override.py:get_permission:1862 > override.py:get_resource:1791:   35
E               dagbag.py:_serialize_dag_capturing_errors:692 > dagbag.py:_sync_perm_for_dag:748 > override.py:sync_perm_for_dag:1123 > override.py:create_permission:1890 > override.py:get_permission:1865:   30

As we included more permissions at the DAG level, we expect an increase in the number of queries during the sync:

airflow/airflow/providers/fab/auth_manager/security_manager/override.py

Lines 1120 to 1123 in 27ba034

	for resource_name, resource_values in self.RESOURCE_DETAILS_MAP.items():
	dag_resource_name = self._resource_name(dag_id, resource_name)
	for dag_action_name in resource_values["actions"]:
	self.create_permission(dag_action_name, dag_resource_name)

Do you think we can increase the query count for this test?

[potiuk]

I think that one caused problem #41540 (comment) - it does not seem backwards-compatible with the previous fab provider.

[kosteev]

It looks like this change implies dependencies between Airflow core and fab provider and new version of fab provider requires changes also in Airflow core, thus fab provider should have constraint apache-airflow>=2.9.X?

It looks like for Airflow 2.9.1 this version of fab provider is not backward compatible

[potiuk]

It looks like for Airflow 2.9.1 this version of fab provider is not backward compatible

@ekosteev - do you see some evidence of this ? The issue here: #41540 (comment) - made the earlier FAB providers not compatible for 2.10.0 -> which shoudl be fixed in #41540, but I have not seen/ compatibility issues with earlier airflow versions (It passed all the compatibility tests at the very least)