Set default wasb Azure http logging level to warning; fixes #16224

[havocbane]

Closes: #16224

Set default Azure wasb http logging level to warning and allow the user to override with an environment variable named "AZURE_HTTP_LOGGING_LEVEL". In the case that a bad level is passed through the environment variable, this will default back to the warning level.

If accepted, please tag the merge with the label hacktoberfest-accepted.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
Here are some useful points:

• Pay attention to the quality of your code (flake8, mypy and type annotations). Our pre-commits will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it’s a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: [email protected]
  Slack: https://s.apache.org/airflow-slack

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

[uranusjr]

Why?

If this configurability is really desired, I’d use airflow.settings for this.

from airflow import settings

logger.setLevel(getattr(settings, "AZURE_HTTP_LOGGING_LEVEL", logging.WARNING))

So a user can add AZURE_HTTP_LOGGING_LEVEL to airflow_local_settings if they so desire. This will need an entry in documentation as well.

With that, I won’t add the ValueError block. If the user provides something that crashes Airflow, that’s the user’s problem and we don’t need to do things too magically.

[havocbane]

I can make those updates and reopen in a new pull request since the bot closed this one. I was mostly thinking the user might want to tweak the log level for these in particular and I was unaware of the standard settings approach. Thanks for the feedback!

[eladkal]

I can re-open if you wish to continue work on it (you will need to rebase to latest main)

[malthe]

This would be really nice to have!

@havocbane did you give up on getting this merged in? I think it's a quite necessary configuration to have to avoid getting overwhelmed with nonsense service logging from the Azure library.

[potiuk]

I also rebased it (just in case)

[boring-cyborg]

Awesome work, congrats on your first merged pull request!