Define version with hatch-vcs

[pquentin]

In order to test our build & publish workflow with Test PyPI (#3411) we need to generate a new package version for every commit. This is what hatch-vcs does by building on top of setuptools-scm. If we're building a tagged version with no local changes, then the tag is used for the verison. Otherwise, versions are like 2.2.3.dev2+g3e13e5a0.d20240618, where:

• 2.2.3.dev2 means two commits after the 2.2.2 tag
• +g3e13e5a0.d20240618 is the current short git commit sha from git describe plus the current date

However, Test PYPI does not accept those local versions with + signs, so we disable them with local_scheme = "no-local-version".

Also, even though the best way to look at package version is with importlib.metadata, we still write a _version.py file on build with __version__ set as it's still used a lot (including by our own CI).

To make sure the wheel built in the emscripten test has the correct version (and not fail when micropip installs requests), I fetch all history. This slows down checkouts by a few seconds at most. Most of them continue to take 1s.

TODO:

• This probably deserves a "Note to distributors" in the changelog even though building will continue to work as before.

[illia-v]

I like the idea!

[sethmlarson]

This seems like a good idea, and I like hatch-vcs's implementation. Two questions:

[sethmlarson]

Does GitHub disallow having a missing file in CODEOWNERS? If not, we could keep the entry to alert maintainers to it being re-added?

[pquentin]

This seems OK. https://github.com/pquentin/urllib3/blob/hatch-vcs/.github/CODEOWNERS reports various errors, but only related to contributors that don't have write access to my fork. Done in 8f18df1 (#3412).

[sethmlarson]

Since hatch-vcs isn't updated often, are we comfortable pinning to a specific version? (ie ==0.4.0)?

[pquentin]

Sure, done in 8f18df1 (#3412). hatch-vcs itself does not pin setuptools-scm, should we do it?

[sigmavirus24]

For what it's worth, gitlint adopted hatch+ hatch-vcs and the CI breakage has been frequent for an otherwise very stable project with far fewer changes than this one. I personally have never had good experiences with it or with setuptools scm but I seem to frequently be in the minority so I wouldn't block this on my feedback. Just know if it starts causing a headache for you, you're not alone

[pquentin]

Hey @sigmavirus24, thanks for the feedback. I'm still on the fence on this one, but it seems like the only way to fix our release automation and not having to end up manually upload releases when doing emergency security fixes.

Do you have specific CI breakages in mind? I did run into one issue already: sparse checkouts done by GitHub Actions mean that building a wheel ends up with a version like 0.0.1 by default. But that's easy to fix.

[sethmlarson]

(FYI, in general I'm also not a fan of VCS-driven versioning, but if hatch-vcs is better than setuptools-scm)

[pquentin]

I haven't managed yet to make coverage ignore the _version.py file, which is breaking coverage combine.

[sigmavirus24]

(FYI, in general I'm also not a fan of VCS-driven versioning, but if hatch-vcs is better than setuptools-scm)

It's based on setuptools scm so not sure how much better it actually is

[illia-v]

I haven't managed yet to make coverage ignore the _version.py file, which is breaking coverage combine.

An empty _version.py seems to do the trick, linting fails though
illia-v@595204f

[pquentin]

Thanks! It's interesting that the lint does not break when the file is entirely missing. To fix it when the file is empty, I suppose we should run mypy against the generated code.

In any case, I'm going to open an issue for the coverage combine failure when the file is missing to see if that's expected behavior.

[pquentin]

I found the fix when trying to submit a minimal example to coverage.py that did not involve GitHub Actions. The thing is that running python -m build or pip install . will write a _version.py in the source tree. So now I run python -m build before combining coverage in GitHub Actions. And everything is green now! (If we want to use illia-v@595204f instead, we simply need to do the same when running lint.)

I guess now we need to decide if we're trying this or not. It does not have to be set in stone, and we can reevaluate in three months if the benefits (testing the release workflow) outweigh the disadvantages. It's not clear to me what the issues with setuptools-scm are.

[illia-v]

Thanks, LGTM!

It does not have to be set in stone, and we can reevaluate in three months if the benefits (testing the release workflow) outweigh the disadvantages. It's not clear to me what the issues with setuptools-scm are.

I agree with this

[sethmlarson]

Let's give it a try, we can always roll back if it's getting in the way.