Convert selective checks to Breeze Python

[potiuk]

Instead of bash-based, complex logic script to perform PR selective
checks we now integrated the whole logic into Breeze Python code.

It is now much simplified, when it comes to algorithm. We've
implemented simple rule-based decision tree. The rules describing
the decision tree are now are now much easier
to reason about and they correspond one-to-one with the rules
that are implemented in the code in rather straightforward way.

The code is much simpler and diagnostics of the selective checks
has also been vastly improved:

• The rule engine displays status of applying each rule and
  explains (with yellow warning message what decision was made
  and why. Informative messages are printed showing the resulting
  output

• List of files impacting the decision are also displayed

• The names of "ci file group" and "test type" were aligned

• Unit tests covering wide range of cases are added. Each test
  describes what is the case they demonstrate

• breeze selective-checks command that is used in CI can also
  be used locally by just providing commit-ish reference of the
  commit to check. This way you can very easily debug problems and
  fix them

Fixes: #19971

---

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragement file, named {pr_number}.significant.rst, in newsfragments.

[potiuk]

This is the next step in Breeze conversion. This time -800 lines of Bash Code (replaced with Python but including lots of unit tests) which is far more readable and easy to reason about.

Plus very clear diagnosticts of what decisions are made and why, including breeze static-check tool that can be used locally to debug and reproduce selective check behaviour with any local commit.

Regular PR that triggers all tests (core change) but does not run any Helm/K8S/UI javascript tests

More complex change (this one) that trigers all possible tests because environment changed:

Fragment of more complex change (amazon + cncf.kubernetes) that triggers only subset of unit tests ("Always", "Providers") bit also all our K8S changes:

Tests are pretty comprehensive and we will be able to easily test and add any new cases in the future (especially when we split providers and want to make more fine-grained decisions which tests to run.

[potiuk]

cc: @edithturn @Bowrna : -800 lines of bash less and much easier "logic" to grasp for selective checks implementation

[edithturn]

I wouldn't have thought of all those changes, really impressive @potiuk.

[potiuk]

I wouldn't have thought of all those changes, really impressive @potiuk.

Just Python is way easier to write things better :)

[potiuk]

BTW. By converting the checks to Python I also found that current selectiuve checks were not "selective enough" as I missed the cases where "Helm" tests are run unnecessarily in quite a number of cases (For example they we run when only providers were modified - this should speed up PRs from contributors who only modified one or more providers and did not touch the core).

[potiuk]

All Green!

[github-actions]

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

[edithturn]

It was merged, well done @potiuk Jarek 🥳