[BUG] Unusually named license_files not included in dist-info

[hroncok]

setuptools version

setuptools===57.4.0

Python version

3.9.6

OS

Fedora Linux 33, x86_64

Additional environment information

wheel===0.36.2

Description

I've tried to test out #2645 but including license files not named according to this pattern:

setuptools/setuptools/dist.py

Line 596 in 0c302d2

patterns = ('LICEN[CS]E*', 'COPYING*', 'NOTICE*', 'AUTHORS*')

Are not included in dist info.

Expected behavior

I expected that files explicitly listed in license_files will always be listed in METADATA under License-File: and will always be included in the dist-info directory regardless of their filename.

How to Reproduce

Consider this setup.py:

from setuptools import setup

setup(
    name='nested_dist',
    version='666',
    license_files=['LICENSE'],
)

And an empty LICENSE file.

Run:

$ python -c 'import setuptools.build_meta; setuptools.build_meta.__legacy__.prepare_metadata_for_build_wheel(".")'

Or:

$ python setup.py bdist_wheel

Or:

$ pip wheel .

You'll see something like:

running dist_info
...
adding license file 'LICENSE'
...
adding license file "LICENSE" (matched pattern "LICEN[CS]E*")
...

The LICENSE file will be listed in METADATA under License-File: and it will be included in the dist-info directory.

Now rename LICENSE to DIFFERENT and adapt license_files accordingly:

from setuptools import setup

setup(
    name='nested_dist',
    version='666',
    license_files=['DIFFERENT'],
)

You'll get:

...
adding license file 'DIFFERENT'
...

The DIFFERENT file will be listed in METADATA under License-File: and it will not be included in the dist-info directory.

Output

See above.

Code of Conduct

• I agree to follow the PSF Code of Conduct

[jaraco]

Agreed that looks like a bug. Thanks for the report.

[jaraco]

@cdce8p Would you be willing to work on a fix?

[cdce8p]

@hroncok Thanks for the report, but this isn't a setuptools issue. The second entry is added by wheel.
Keep in mind that for wheel to be able to add license files based on patterns, you need to add license_files to the [metadata] section in setup.cfg, not define in in setup(). That will also add the license file to the .dist-info folder, once it's matched. For details, please refer to the wheel documentation: https://wheel.readthedocs.io/en/stable/user_guide.html#including-license-files-in-the-generated-wheel-file

As for the License-File: field in the METADATA: This one isn't in the official packaging spec yet. We only added it because it is useful for other project to be able and easily find the license files form the metadata. There is an effort to make it official, but it will take some more time. Until then, I don't expect wheel to support it.
https://www.python.org/dev/peps/pep-0639/#license-file-multiple-use

[hroncok]

I kinda expected that setuptools supports PEP 639 before it is accepted, as a "preview". I understand that the PEP is still a draft.

I don't understand what is the point of listing missing files as License-File. If wheel cannot add files like this, setuptools should not list them and fail instead.

[hroncok]

With setup.cfg, it kinda works.

However, when the licenses are in nested directories, as listed in the given example: https://wheel.readthedocs.io/en/stable/user_guide.html#including-license-files-in-the-generated-wheel-file

[metadata]
license_files =
   3rdparty/*.txt`

The license files are included directly in the dist-info directory (e.g. as license.txt) and the License-File: entries are incorrect (e.g. 3rdparty/license.txt).

[cdce8p]

I don't understand what is the point of listing missing files as License-File. If wheel cannot add files like this, setuptools should not list them and fail instead.

If you're referring to the license_files argument in setup(), I agree this should be deprecated / removed.

With setup.cfg, it kinda works.

However, when the licenses are in nested directories, as listed in the given example: https://wheel.readthedocs.io/en/stable/user_guide.html#including-license-files-in-the-generated-wheel-file

The license files are included directly in the dist-info directory (e.g. as license.txt) and the License-File: entries are incorrect (e.g. 3rdparty/license.txt).

That is still a point that needs to be addressed. IMO wheel needs to update their implementation here. Although they do mention the behavior in their docs, it can cause name conflicts, e.g., if a file named LICENSE is present in a sub dir in addition to the main one.

No matter the path, all the matching license files are written in the wheel in the .dist-info
directory based on their file name only.

In that case 3rdparty/LICENSE would overwrite the previous one.

--
I'm open to suggestions if you have an idea how to better resolve this conflict.

[encukou]

Also, what happens if someone names the license file RECORD?

Raise an error if an existing file in dist-info would be overwritten? That'd allow adding better solutions in the future.

[abravalheri]

The original problem seems to be already solved with the latest versions of setuptools and wheel:

> docker run --rm -it python:3.11 /bin/bash


mkdir /tmp/myproj
cd /tmp/myproj
cat <<EOF > setup.py
from setuptools import setup

setup(
    name='myproj',
    version='666',
    license_files=['DIFFERENT'],
)
EOF
touch DIFFERENT
python -m pip install 'setuptools==66.0.0' 'wheel==0.38.4'
python -c 'import setuptools.build_meta; setuptools.build_meta.__legacy__.prepare_metadata_for_build_wheel(".")'
ls myproj-666.dist-info  # => DIFFERENT  METADATA  top_level.txt
rm -rf myproj-666.dist-info
python -c 'import setuptools.build_meta; setuptools.build_meta.prepare_metadata_for_build_wheel(".")'
ls myproj-666.dist-info  # => DIFFERENT  METADATA  top_level.txt
python -m pip install -U build
python -m build
unzip -l dist/*.whl
# Archive:  dist/myproj-666-py3-none-any.whl
#   Length      Date    Time    Name
# ---------  ---------- -----   ----
#         0  2023-01-20 10:48   myproj-666.dist-info/DIFFERENT
#        73  2023-01-20 10:48   myproj-666.dist-info/METADATA
#        92  2023-01-20 10:48   myproj-666.dist-info/WHEEL
#         1  2023-01-20 10:48   myproj-666.dist-info/top_level.txt
#       367  2023-01-20 10:48   myproj-666.dist-info/RECORD
# ---------                     -------
#       533                     5 files

I think that for the problems regarding name clashes and etc we should wait for PEP 639 implementation.