Replace datetime.datetime.utcnow by airflow.utils.timezone.utcnow in core

[Taragolis]

This one deprecated in Python 3.12, see What’s New In Python 3.12: Deprecated

datetime: datetime.datetime’s utcnow() and utcfromtimestamp() are deprecated and will be removed in a future version. Instead, use timezone-aware objects to represent datetimes in UTC: respectively, call now() and fromtimestamp() with the tz parameter set to datetime.UTC. (Contributed by Paul Ganssle in gh-103857.)

We still far away of support Python 3.12

Related: #32344

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

[Taragolis]

Finally I have a time to resolve all usage of datetime.datetime.utcnow (deprecated in Python 3.12) and pendulum.DateTime.utcnow (just tried to reduce direct usage of pendulum) into the core and resolve all failing tests.

[potiuk]

Nice. Those were also causing max recursion errors in the Python 3.12 PR

[Taragolis]

I guess this changes also could make some internal components more consistent, because datetime.datetime.utcnow return naive object and at least into the some tests it makes it failure if local timezone non UTC

[Taragolis]

There is still remaining usage into the dev and providers tests, which I've plan to fix as follow up.
And after that we could ban datetime.datetime.utcnow import and usage on entire project by add into the tool.ruff.flake8-tidy-imports.banned-api rule

[tool.ruff.flake8-tidy-imports.banned-api]                                                                         
...      
"datetime.datetime.utcnow".msg = "Use airflow.utils.timezone.utcnow or datetime.datetime.utc(tz=...)"

❯ pre-commit run ruff --all-files
Run 'ruff' for extremely fast Python linting.............................Failed
- hook id: ruff
- exit code: 1

airflow/providers/google/cloud/transfers/s3_to_gcs.py:279:15: TID251 `datetime.datetime.utcnow` is banned: Use airflow.utils.timezone.utcnow or datetime.datetime.utc(tz=...)
airflow/providers/google/common/hooks/base_google.py:673:41: TID251 `datetime.datetime.utcnow` is banned: Use airflow.utils.timezone.utcnow or datetime.datetime.utc(tz=...)
dev/breeze/src/airflow_breeze/utils/parallel.py:286:27: TID251 `datetime.datetime.utcnow` is banned: Use airflow.utils.timezone.utcnow or datetime.datetime.utc(tz=...)
dev/breeze/src/airflow_breeze/utils/parallel.py:315:23: TID251 `datetime.datetime.utcnow` is banned: Use airflow.utils.timezone.utcnow or datetime.datetime.utc(tz=...)
kubernetes_tests/test_base.py:80:32: TID251 `datetime.datetime.utcnow` is banned: Use airflow.utils.timezone.utcnow or datetime.datetime.utc(tz=...)
scripts/in_container/update_quarantined_test_status.py:239:26: TID251 `datetime.datetime.utcnow` is banned: Use airflow.utils.timezone.utcnow or datetime.datetime.utc(tz=...)
Found 6 errors.
tests/providers/celery/executors/test_celery_executor.py:186:22: TID251 `datetime.datetime.utcnow` is banned: Use airflow.utils.timezone.utcnow or datetime.datetime.utc(tz=...)
tests/providers/cncf/kubernetes/executors/test_kubernetes_executor.py:453:37: TID251 `datetime.datetime.utcnow` is banned: Use airflow.utils.timezone.utcnow or datetime.datetime.utc(tz=...)
tests/providers/cncf/kubernetes/executors/test_kubernetes_executor.py:1420:48: TID251 `datetime.datetime.utcnow` is banned: Use airflow.utils.timezone.utcnow or datetime.datetime.utc(tz=...)
tests/providers/cncf/kubernetes/executors/test_kubernetes_executor.py:1450:48: TID251 `datetime.datetime.utcnow` is banned: Use airflow.utils.timezone.utcnow or datetime.datetime.utc(tz=...)
tests/providers/cncf/kubernetes/executors/test_kubernetes_executor.py:1484:48: TID251 `datetime.datetime.utcnow` is banned: Use airflow.utils.timezone.utcnow or datetime.datetime.utc(tz=...)
tests/providers/google/cloud/sensors/test_gcs.py:75:37: TID251 `datetime.datetime.utcnow` is banned: Use airflow.utils.timezone.utcnow or datetime.datetime.utc(tz=...)
tests/providers/google/cloud/triggers/test_gcs.py:44:18: TID251 `datetime.datetime.utcnow` is banned: Use airflow.utils.timezone.utcnow or datetime.datetime.utc(tz=...)
tests/system/providers/google/cloud/storage_transfer/example_cloud_storage_transfer_service_aws.py:90:29: TID251 `datetime.datetime.utcnow` is banned: Use airflow.utils.timezone.utcnow or datetime.datetime.utc(tz=...)
tests/system/providers/google/cloud/storage_transfer/example_cloud_storage_transfer_service_gcp.py:86:29: TID251 `datetime.datetime.utcnow` is banned: Use airflow.utils.timezone.utcnow or datetime.datetime.utc(tz=...)
Found 9 errors.