Change logging level details of connection info in get_connection()

[josh-fell]

Related: #19883

Currently task logs can contain all of connection details depending on how the associated connection to the task is configured (i.e. if host is a provided connection attr). These details are logged at the INFO level but seem more appropriate for debugging.

This PR intends to clean up this connection logging a little. The INFO level logging will contain only the connection ID that is used while the details of the connection are changed to the DEBUG level (and still masked). Additionally the connection ID info is logged regardless of the provided connection attrs (i.e. removing the host check). Lastly this change also has a small added benefit of not accidentally or unknowingly exposing connection info that users do not want in their logs first rather than the details be exposed and then having to setup configuration to mask them later (assuming the exposure is noticed at all).

---

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

[github-actions]

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

[josh-fell]

Is it expected that the Providers tests in the "Tests: Always API Core Other CLI Providers Integration" suite doesn't run for MySQL and MSSQL? I was surprised that only Postgres and Sqlite failed.

[potiuk]

Is it expected that the Providers tests in the "Tests: Always API Core Other CLI Providers Integration" suite doesn't run for MySQL and MSSQL? I was surprised that only Postgres and Sqlite failed.

Yes. That was done as part of stabilizing our flaky CI Tests.

Both MySQL and MSSQL (despite very aggressive optimisation of the configuration of the dockerized versions of those) require much more memory to run than Postgres and SQLite. That lead to Jobs failing quite often with Exit code 137 (means memory run out) when they were run on Public Runners. That's why this type of tests is disabled now for those two databases (but only on Public Runners).

In fact, it is actually even printed there. If you unfold "Determine how to run the tests" in those tests you will see this:

You will find the logic controlling it here

airflow/scripts/ci/testing/ci_run_airflow_testing.sh

Line 68 in 906d710

function run_all_test_types_in_parallel() {

Also "all" tests will run in "main" after the change is merged. Those tests are run on our 64GB mem self-hosted machines, that have enough CPUS and memory to run all the tests always in parallell. So we will see failing main in case those "Providers" tests run on MySQL or MsSQL woudl fail (which is highly unlikely because Provider tests are not supposed to be "Metadata-DB" dependent. Yet we "just in case" always run all tests in main.

[potiuk]

The same tests on self-hosted runners look like that:

[josh-fell]

Oh that makes perfect sense. Thanks for all the context @potiuk 🚀

[potiuk]

Oh that makes perfect sense. Thanks for all the context @potiuk 🚀

No problem. I am writing all those "Architecture decision records" in https://github.com/apache/airflow/tree/main/dev/breeze/doc/adr - during the Breeze2 rewrite project, so this is actually a good next one to add.

[potiuk]

Just old docker-compose problem already fixed. Merging