Poetry pip shims fix and dropping py3.7 in the same release breaks isort on py3.7 without recourse

[vedantpuri]

The following PRs were bundled in 5.12.0

• Fix poetry pip-shims extras dependency #2078
• Removal Python 3.7 #2064

If I understand this correctly, anyone using isort on py3.7 does not get the poetry fix. Should there have been an intermediary release before 5.12.0 to cover this case ?

[sijunhe]

+1 to this. We are also in the same dilemma. Would it be possible to do another release that fixes isort for py3.7?

[noorul]

Looks like the release 5.12.0 has dropped support for 3.7 and this has put everyone using < 3.8 in dilemma.

[psdon]

This also affects python 3.11

[exhuma]

It also seems to affect Python 3.10

We have over 100 projects, each of which with their own .pre-commit-config.yml with daily dev-builds configured. This caused a ton of error-notifications being generated this morning.

Updating all of the projects would be pretty cumbersome. But without a fix we'll be confronted with the same notification mess tomorrow (and each day after).

We can mitigate this by pinning an isort version in a centrally shared part of the pipeline. But that's pretty hacky and I would appreciate a fix by isort considering that this has been affecting a lot of people.

[RobPasMue]

This also affects python 3.11

We are having issues throughout many repositories because of this problem as well on Python 3.11...

[timothycrosley]

@psdon @exhuma and @RobPasMue Can you confirm you are using the latest version of isort (5.12.0)? This issue should be fixed in that release, but is bundled with the project deprecating support for Python3.7. I was planning on making a hot fix for 3.7 with the workaround for the surprising breaking change from poetry, but if this is still happening with the fix in place on those newer versions more investigation will be needed. Is is possible all these Python3.10/3.11 projects have the old version of isort pinned, or need to run a pre-commit autoupdate? If that's the case there's nothing I could do at the isort level to fix that, since any changes I make will be new releases.

[RobPasMue]

That's true, I was pointing to 5.11.4 while using Python 3.11 - thanks for pointing it out @timothycrosley! That solved our issues =)

[exhuma]

@timothycrosley The projects are not running on isort 5.12 and updating all of the projects is not feasible. There just are too many and we cannot drop everything right now to work on this. It's too time intensive. We need to go through all the failures, check out the code, run autoupdate commit and push for over 100 projects.

Running autoupdate can also have unintended side effects as it updates everything. We've had surprising failures in the past with projects like mypy when it suddenly started detecting new type-errors in newer versions. We just cannot risk running this "blindly" on every project.

This will be done, but we cannot do this quickly enough and I prefer to do this in a calm, controlled space.

We have a shared CI pipeline which gives me some flexibility but the nature of pre-commit is that the config is inside the project. So each project needs to be touched individually.

The quickest thing we can do is to disable isort everywhere. But I'm working on a more invasive solution which would not require us to drop isort. I will share it as soon as it's done (which is soon) ;)

[timothycrosley]

@exhuma best of luck on your approach to fix this! I really wish there was a way for me to safely retroactively fix the old release, but it simply isn't possible. The only project that could fix it for existing references to the old version would be the poetry project, where the incompatibility was introduced

[exhuma]

Here is a script that I will shim into our shared pipeline runs. It's far from ideal, but it allows us to go through our projects piece by piece while keeping the pipelines "green". For anyone else who needs this, feel free to use it.

This script replaces all isort references with 5.12.0. So even newer versions of isort would be "downgraded" which is pretty bad. But it's a workaround.

#!/usr/bin/env python
"""
Fix for https://github.com/PyCQA/isort/issues/2083

This script *modifies* ``.pre-commit-config.yaml`` and pins the isort version
to 5.12.0

This is a hacky workaround for the aforementioned GitHub issue. Considering
that this aggressively changes the revision number for ``isort`` this should
only be used as a workaround if no other solution is available.

It should also be removed as soon as it is no longer needed because this will
"undo" any upgrades to ``isort`` in repositories.

This script assumes to be run in a CI-pipeline. As such, it also assumes that
it can modify files with impunity as they will not be committed back to the
source-code and will be lost after CI pipeline cleanup. This is - in this case
- the intended behaviour.



--- LICENSE ------------------------------------------------------------------

Copyright 2023 Michel Albert

Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
"""

from copy import deepcopy
from os.path import exists
from shutil import copyfile
from textwrap import dedent

from yaml import dump, safe_load

import yamlfix.services

source_file = ".pre-commit-config.yaml"
backup_file = f"{source_file}.bak"

if exists(backup_file):
    raise FileExistsError(f"File {backup_file} already exists")

copyfile(source_file, backup_file)

with open(source_file, encoding="utf8") as fptr:
    content = safe_load(fptr)

replacement_done = False
for repo in content.get("repos", []):
    if repo["repo"] == "https://github.com/PyCQA/isort" and not repo["rev"].startswith(
        "5.12"
    ):
        original = deepcopy(repo)
        repo["rev"] = "5.12.0"
        template = dedent(
            f"""\
            !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
                Problematic isort version {original["rev"]} detected
            !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
            Replacing with pinned 5.12.0
            This is a *temporary* fix. For more information see

            https://github.com/PyCQA/isort/issues/2083
            ----------------------------------------------------------------
            Original Config
            {{original}}
            ----------------------------------------------------------------
            New Config
            {{new}}
            !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
            """
        )
        print(
            template.format(
                original=yamlfix.fix_code(dump(original)),
                new=yamlfix.fix_code(dump(repo)),
            )
        )
        print(f"Updating {repo} with pinned isort version")
        print(f"Updated to {repo}")
        replacement_done = True

with open(source_file, "w", encoding="utf8") as fptr:
    output = yamlfix.services.fix_code(dump(content))
    fptr.write(output)

[exhuma]

The only project that could fix it for existing references to the old version would be the poetry project, where the incompatibility was introduced

Any change to prod the poetry devs for this? As a member of the isort project you would surely have more weight than myself 😉

[timothycrosley]

Closing this as I've fixed this for 3.7 as well with a hotfix: 5.11.5