[flake8-pyi] Implement autofix for redundant-numeric-union (PYI041)

[sbrugman]

Summary

This PR adds autofix for redundant-numeric-union (PYI041)

There are some comments below to explain the reasoning behind some choices that might help review.

Resolves part of #14185.

Test Plan

[github-actions]

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

ℹ️ ecosystem check detected linter changes. (+0 -4 violations, +66 -0 fixes in 3 projects; 51 projects unchanged)

apache/airflow (+0 -0 violations, +58 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview --select ALL

- airflow/decorators/__init__.pyi:117:25: PYI041 Use `float` instead of `int | float`
+ airflow/decorators/__init__.pyi:117:25: PYI041 [*] Use `float` instead of `int | float`
- airflow/decorators/__init__.pyi:256:25: PYI041 Use `float` instead of `int | float`
+ airflow/decorators/__init__.pyi:256:25: PYI041 [*] Use `float` instead of `int | float`
- airflow/jobs/job.py:308:39: PYI041 Use `float` instead of `int | float`
+ airflow/jobs/job.py:308:39: PYI041 [*] Use `float` instead of `int | float`
- airflow/metrics/base_stats_logger.py:39:15: PYI041 Use `float` instead of `int | float`
+ airflow/metrics/base_stats_logger.py:39:15: PYI041 [*] Use `float` instead of `int | float`
- airflow/metrics/base_stats_logger.py:50:15: PYI041 Use `float` instead of `int | float`
+ airflow/metrics/base_stats_logger.py:50:15: PYI041 [*] Use `float` instead of `int | float`
- airflow/metrics/base_stats_logger.py:61:15: PYI041 Use `float` instead of `int | float`
+ airflow/metrics/base_stats_logger.py:61:15: PYI041 [*] Use `float` instead of `int | float`
- airflow/metrics/datadog_logger.py:103:16: PYI041 Use `float` instead of `int | float`
+ airflow/metrics/datadog_logger.py:103:16: PYI041 [*] Use `float` instead of `int | float`
- airflow/metrics/otel_logger.py:237:16: PYI041 Use `float` instead of `int | float`
+ airflow/metrics/otel_logger.py:237:16: PYI041 [*] Use `float` instead of `int | float`
- airflow/metrics/statsd_logger.py:117:16: PYI041 Use `float` instead of `int | float`
+ airflow/metrics/statsd_logger.py:117:16: PYI041 [*] Use `float` instead of `int | float`
- airflow/utils/timezone.py:240:26: PYI041 Use `float` instead of `int | float`
+ airflow/utils/timezone.py:240:26: PYI041 [*] Use `float` instead of `int | float`
- airflow/utils/timezone.py:305:16: PYI041 Use `float` instead of `int | float`
+ airflow/utils/timezone.py:305:16: PYI041 [*] Use `float` instead of `int | float`
- performance/src/performance_dags/performance_dag/performance_dag_utils.py:175:27: PYI041 Use `float` instead of `int | float`
+ performance/src/performance_dags/performance_dag/performance_dag_utils.py:175:27: PYI041 [*] Use `float` instead of `int | float`
- performance/src/performance_dags/performance_dag/performance_dag_utils.py:337:31: PYI041 Use `float` instead of `int | float`
+ performance/src/performance_dags/performance_dag/performance_dag_utils.py:337:31: PYI041 [*] Use `float` instead of `int | float`
- providers/src/airflow/providers/amazon/aws/hooks/batch_client.py:273:16: PYI041 Use `float` instead of `int | float`
+ providers/src/airflow/providers/amazon/aws/hooks/batch_client.py:273:16: PYI041 [*] Use `float` instead of `int | float`
- providers/src/airflow/providers/amazon/aws/hooks/batch_client.py:302:56: PYI041 Use `float` instead of `int | float`
+ providers/src/airflow/providers/amazon/aws/hooks/batch_client.py:302:56: PYI041 [*] Use `float` instead of `int | float`
- providers/src/airflow/providers/amazon/aws/hooks/batch_client.py:325:57: PYI041 Use `float` instead of `int | float`
+ providers/src/airflow/providers/amazon/aws/hooks/batch_client.py:325:57: PYI041 [*] Use `float` instead of `int | float`
- providers/src/airflow/providers/amazon/aws/hooks/batch_client.py:516:27: PYI041 Use `float` instead of `int | float`
+ providers/src/airflow/providers/amazon/aws/hooks/batch_client.py:516:27: PYI041 [*] Use `float` instead of `int | float`
- providers/src/airflow/providers/amazon/aws/hooks/batch_client.py:516:47: PYI041 Use `float` instead of `int | float`
+ providers/src/airflow/providers/amazon/aws/hooks/batch_client.py:516:47: PYI041 [*] Use `float` instead of `int | float`
- providers/src/airflow/providers/amazon/aws/hooks/batch_client.py:516:72: PYI041 Use `float` instead of `int | float`
+ providers/src/airflow/providers/amazon/aws/hooks/batch_client.py:516:72: PYI041 [*] Use `float` instead of `int | float`
- providers/src/airflow/providers/amazon/aws/hooks/batch_client.py:544:22: PYI041 Use `float` instead of `int | float`
+ providers/src/airflow/providers/amazon/aws/hooks/batch_client.py:544:22: PYI041 [*] Use `float` instead of `int | float`
- providers/src/airflow/providers/amazon/aws/hooks/batch_waiters.py:195:16: PYI041 Use `float` instead of `int | float`
... 17 additional changes omitted for project

apache/superset (+0 -0 violations, +8 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview --select ALL

- superset/daos/query.py:68:52: PYI041 Use `float` instead of `int | float`
+ superset/daos/query.py:68:52: PYI041 [*] Use `float` instead of `int | float`
- superset/utils/cache.py:150:14: PYI041 Use `float` instead of `int | float`
+ superset/utils/cache.py:150:14: PYI041 [*] Use `float` instead of `int | float`
- superset/utils/core.py:349:24: PYI041 Use `float` instead of `int | float`
+ superset/utils/core.py:349:24: PYI041 [*] Use `float` instead of `int | float`
- superset/utils/decorators.py:163:24: PYI041 Use `float` instead of `int | float`
+ superset/utils/decorators.py:163:24: PYI041 [*] Use `float` instead of `int | float`

python/typeshed (+0 -4 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview --select E,F,FA,I,PYI,RUF,UP,W

- stdlib/random.pyi:45:31: PYI041 Use `float` instead of `int | float`
- stdlib/random.pyi:52:27: PYI041 Use `float` instead of `int | float`
- stdlib/turtle.pyi:443:16: PYI041 Use `float` instead of `int | float`
- stdlib/turtle.pyi:444:17: PYI041 Use `float` instead of `int | float`

Changes by rule (1 rules affected)

code	total	+ violation	- violation	+ fix	- fix
PYI041	70	0	4	66	0

[MichaReiser]

Thanks. This overall looks great.

The main open questions are

• should use text based edits to avoid reformatting unchanged code and loosing inner comments.
• Whether it's possible to always mark the fix as safe, regardless of comments

[MichaReiser]

Oh, and did you look through the ecosystem changes? If so, could you update the test plan. It's helpful to know if you did because I then don't need to carefully review all of them and can instead just click through some of them.

[sbrugman]

Oh, and did you look through the ecosystem changes? If so, could you update the test plan. It's helpful to know if you did because I then don't need to carefully review all of them and can instead just click through some of them.

The ecosystem results look good.
From this example I'm wondering if we shouldn't include return types; it appears to skip that one now.

Typeshed shows negative in the ecosystem results, but I expect that to not be related to the PR.
Maybe @AlexWaygood can help out with that.
It's probably because the example is Py3.13+

[MichaReiser]

From this example I'm wondering if we shouldn't include return types; it appears to skip that one now.

Could you expand what you mean by it appears to skip that now. Is it not flagging this case or do you think this case shouldn't be flagged?

Is it worth to add a test like that example (or that specific example) to our test suite?

[MichaReiser]

Reading the documentation, I get the impression the fix isn't safe because it flattens nested union expressions. But I understand that this doesn't make the fix unsafe, it's just an extension where the fix goes beyond what one might expect.

[sbrugman]

Exactly. I've reused the phrasing that Charlie used for another rule.

The nesting is a feature to allow cases such as these:

ReadOnlyMode         = Literal["r", "r+"]
WriteAndTruncateMode = Literal["w", "w+", "wt", "w+t"]
WriteNoTruncateMode  = Literal["r+", "r+t"]
AppendMode           = Literal["a", "a+", "at", "a+t"]

AllModes = Literal[ReadOnlyMode, WriteAndTruncateMode,
                   WriteNoTruncateMode, AppendMode]

As a consequence this redundant nesting is also allowed, but can be safely fixed:

AllModes = Literal[Literal["r", "r+"], Literal["w", "w+", "wt", "w+t"],
                   Literal["r+", "r+t"], Literal["a", "a+", "at", "a+t"]]

See https://typing.readthedocs.io/en/latest/spec/literal.html#legal-and-illegal-parameterizations

We probably should have I will create a rule for this (for both union and literal).

[MichaReiser]

After reading the linked document, I understand that both examples are valid. Can you share an example where flattening a nested union is unsafe?

[charliermarsh]

I tweaked the comment a bit on all three rules.

[AlexWaygood]

Typeshed shows negative in the ecosystem results, but I expect that to not be related to the PR.
Maybe @AlexWaygood can help out with that.
It's probably because the example is Py3.13+

The ecosystem reports for typeshed have often not been reproducible locally and nobody's yet had a chance to look into why... if you can't repro the typeshed hits locally, I'd just ignore them!

[sbrugman]

Could you expand what you mean by it appears to skip that now. Is it not flagging this case or do you think this case shouldn't be flagged?

After investigating this, it turned out that this rule hooked into statement.rs instead of expression.rs like all the other rules related to unions do. This caused false negatives on the return types and type aliases. We had no test for this. Fixed it, and added a test.