dash vs underscore in wheel and sdist metadata name

[wwuck]

I have a package with the name defined in setup.cfg:

[metadata]
name = my_package
version = 0.0.1

I am building the package with python3 -m build.

When I inspect the generated artifacts both wheel and sdist filenames contain my_package-0.0.1 in the filename, however I see the wheel and sdist have different names defined in the included metadata.

The wheel has in METADATA:

Metadata-Version: 2.1
Name: my-package
Version: 0.0.1

The sdist has in PKG-INFO:

Metadata-Version: 2.1
Name: my_package
Version: 0.0.1

When I upload these two files to my private Nexus repository, they each appear in a separate location.
The wheel ends up at my-package/0.0.1/my_package-0.0.1-py3-none-any.whl while the sdist ends up at my_package/0.0.1/my_package-0.0.1.tar.gz.

If I change the name in setup.cfg from my_package to my-package then the wheel and sdist now have different filenames (my_package-0.0.1-py3-none-any.whl vs. my-package-0.0.1.tar.gz) but they do now have identical metadata Name: my-package.

Are the wheel and sdist formats are supposed to generate different metadata/filenames from the same source and I should submit a bug report to Nexus?

Or they should contain identical metadata/filenames and it is a bug in build?

Or are both build and Nexus operating correctly following any relevant PEPs and there's nothing to fix here?

[wwuck]

I should have mentioned, this was with the setuptools backend in pyproject.toml.

I also tested with flit and that enforces _ in the project name field in pyproject.toml.

flit also generates identical naming for _ everywhere in both wheel and sdist filenames, and in the wheel and sdist metadata Name fields, so setuptools is operating differently here.

[uranusjr]

This is specific to the backend and not related to build. You should talk to either setuptools or flit if you find the behaviour problematic. But the behaviour is purely cosmetic and should not be problematic in any way.

Technical context: The behaviour is called name normalisation. Underscore and dash (also dot) are treated as equivalent in Python projects and therefore used interchangably in a lot of places. For example, the wheel always use _ because using - in the filename would make parsing ambiguous. An sdist can technically use either (the rightmost dash separates the name and version aprts); underscore is preferred for consistency with wheels, but setuptools use dash for backward compatibility reasons (its behaviour outdates wheels). So the only issue here is whether the name should be normalised in the Name metadata. Since setuptools allows putting unnormlised names in the specification (setup.py), it just copies that user specification unchanged and performs normalisation when needed (i.e. for the wheel's name). In Flit, however, it is not possible to use dash and dot because the name is also used to identify the import location, so it always normalises for consistency.

[wwuck]

I did realise after seeing the different behaviour from flit, I should have filed this issue under setuptools instead of build. Thanks for the explanation.

[wwuck]

Is this normalised Name in metadata an issue for the specific backend too or is that handled in build? If so I can submit an issue to setuptools.

[wwuck]

pypa/setuptools#2522

[uranusjr]

It's also generated by the backend. I don't think setuptools would be interested in changing this though, also mainly for compatibility reasons. The packaging standard also does not require backends to perform normalisation there, so it's well within their rights to not normalise. Furthermore IIRC a setuptools maintainer (forgot who) once told me they prefer keeping unnormalised name in package metadata because that's what the user wants to name their package. I don't particularly agree with it, but again, they are allowed to do what they wish here.

[layday]

https://packaging.python.org/specifications/source-distribution-format/#source-distribution-file-name says, "The name and version components of the [sdist] filename MUST match the values stored in the metadata contained in the file", where "the file" presumably means PKG-INFO. I don't know if this requirement is backed by a PEP or where or when it was decided.

[layday]

Okay, PEP 643 contains this line:

In addition, this PEP makes the PyPA Specifications [2] document the canonical location for the specification of the source distribution format (collecting the information in PEP 517 and in this PEP).

... so whatever that document says goes, though I can't imagine that it should apply retroactively, so setuptools would have to embrace Core Metadata 2.2 first...

[uranusjr]

My interpretation to "match" in the sentence is the name (and version) should match metadata after normalisation, not that they must match literally. But wording (among many other things) around sdist has always been very lax so feel free to raise this issue on Discourse for clarification and perhaps a fix to the standard.

[layday]

That would be very peculiar if true but I've started https://discuss.python.org/t/source-distribution-specification-clarification/10965. Closing this one - you can follow the Discourse thread and the setuptools issue for the issue with setuptools.