[BUG] license_files not showing up on wheel if not set on setup.cfg (setup.py & pyproject.toml).

[ashb]

setuptools version

setuptools==65.2.0

Python version

Python 3.10.7

OS

Arch Linux

Additional environment information

No response

Description

A setuptools based distribtuion does not include the files specified in license-files under the tool.setuptools table in the wheel files dist-info (by a setup.py/setup.cfg) based project does).

Expected behavior

It works for sdist, and I see appropriate lines about "adding license-A.txt" but it doesn't show up in the wheel when pyproject.toml is present

How to Reproduce

With pyproject.toml

mkdir -p licenses
date > licenses/license-A.txt
cat > pyproject.toml << EOF
[build-system]
requires = ['setuptools==65.2.0']
build-backend = "setuptools.build_meta"

[project]
name = "repro-case"
version = '2.5.0dev0'
requires-python = "~=3.7"
license = { text = "Apache License 2.0" }
authors = [
    { name = "Ash", email = "[email protected]" },
]
[tool.setuptools]
license-files = [
    'LICENSE',
    'licenses/license-A.txt',
]
EOF
rm -r repro_case.egg-info/ dist
python -m build
unzip -l dist/*.whl

With out pyproject.toml:

mkdir -p licenses
date > licenses/license-A.txt
cat > setup.cfg << EOF
[metadata]
name = repro-case
version = 2.5.0dev0
requires_python = ~=3.7
author = Apache Software Foundation
author_email = [email protected]
license_files =
    LICENSE
    licenses/license-A.txt
EOF
cat > setup.py <<EOF
import setuptools
setuptools.setup()
EOF
rm -r repro_case.egg-info/ dist
python -m build
unzip -l dist/*.whl

Output

Expected output (and this is what I get with setup.cfg includes repro_case-2.5.0.dev0.dist-info/license-A.txt

$ unzip -l dist/*.whl 
Archive:  dist/repro_case-2.5.0.dev0-py3-none-any.whl
  Length      Date    Time    Name
---------  ---------- -----   ----
      169  2022-09-20 16:04   repro_case-2.5.0.dev0.dist-info/METADATA
       92  2022-09-20 16:04   repro_case-2.5.0.dev0.dist-info/WHEEL
       32  2022-09-20 16:04   repro_case-2.5.0.dev0.dist-info/license-A.txt
        9  2022-09-20 16:04   repro_case-2.5.0.dev0.dist-info/top_level.txt
      428  2022-09-20 16:04   repro_case-2.5.0.dev0.dist-info/RECORD
---------                     -------
      730                     5 files

Output with pyproject.toml does not include extra file

 unzip -l dist/*.whl                                                                                                                                                                                  [17:22] ✓ !1161
Archive:  dist/repro_case-2.5.0.dev0-py3-none-any.whl
  Length      Date    Time    Name
---------  ---------- -----   ----
      205  2022-09-20 16:22   repro_case-2.5.0.dev0.dist-info/METADATA
       92  2022-09-20 16:22   repro_case-2.5.0.dev0.dist-info/WHEEL
        1  2022-09-20 16:22   repro_case-2.5.0.dev0.dist-info/top_level.txt
      328  2022-09-20 16:22   repro_case-2.5.0.dev0.dist-info/RECORD
---------                     -------
      626                     4 files

[ashb]

I've done a tiny bit of debugging, and after _apply_pyprojecttoml.py's apply function has run dist.metadata.license_files has the values in it. I'm not yet familiar with setuptools to know where to look next as to what the problem might be.

[abravalheri]

Hi @ashb it would seem that configurations in pyproject.toml have the same effect as configurations in setup.py (instead of setup.cfg). I tried the following:

mkdir /tmp/test-licenses
cd /tmp/test-licenses
mkdir -p licenses
date > licenses/license-A.txt
cat > pyproject.toml << EOF
[build-system]
requires = ['setuptools==65.2.0']
build-backend = "setuptools.build_meta"
EOF
cat > setup.py <<EOF
from setuptools import setup

setup(
    name="repro-case",
    version='2.5.0dev0',
    python_requires="~=3.7",
    license="Apache License 2.0",
    author="Ash <[email protected]>",
    license_files=[
        'LICENSE',
        'licenses/license-A.txt',
    ],
)
EOF
rm -r repro_case.egg-info/ dist
pipx run build
unzip -l dist/*.whl

which results in the following:

...
Successfully built repro-case-2.5.0.dev0.tar.gz and repro_case-2.5.0.dev0-py3-none-any.whl
Archive:  dist/repro_case-2.5.0.dev0-py3-none-any.whl
  Length      Date    Time    Name
---------  ---------- -----   ----
      176  2022-09-20 17:46   repro_case-2.5.0.dev0.dist-info/METADATA
       92  2022-09-20 17:46   repro_case-2.5.0.dev0.dist-info/WHEEL
        9  2022-09-20 17:46   repro_case-2.5.0.dev0.dist-info/top_level.txt
      328  2022-09-20 17:46   repro_case-2.5.0.dev0.dist-info/RECORD
---------                     -------
      605                     4 files

Of course, this is equally undesirable...

I think what is happening is the following:

• pypa/wheel (which is the separated project that specifies the bdist_wheel command used by setuptools to create wheels, gets the list of license files using self.distribution.get_option_dict("metadata") instead of self.distribution.metadata.
• Unfortunately this "option dict" is populated exclusively from the setup.cfg-style files + command line arguments.

https://github.com/pypa/wheel/blob/da6752626693b46ad0a5c4dce59624a5223124f7/src/wheel/bdist_wheel.py#L434

I have to think a little bit more about this (if there is something reasonable that we can do in the setuptools side, without requiring changes in wheel).

[abravalheri]

Probably the same underlying problem as #2739.

[ashb]

Oh right. Interesting! I guess we had setup.cfg in our project before we added license_files

[ashb]

An option would be to put that specific thing in the command-options metadata section. It's not even against the documented purpose of command_options from distutils, specifically the "from config files" bit of the below comment:

        # 'command_options' is where we store command options between
        # parsing them (from config files, the command-line, etc.) and when
        # they are actually needed -- ie. when the command in question is
        # instantiated.  It is a dictionary of dictionaries of 2-tuples:
        #   command_options = { command_name : { option : (source, value) } }

(I agree that it's not a great option overall though)

Forgive me ex-BDFL, for I have sinned:

from wheel.bdist_wheel import bdist_wheel

class Wheel(bdist_wheel):
    def finalize_options(self):
        super().finalize_options()
        metadata_opt = self.distribution.get_option_dict('metadata')
        metadata_opt['license_files'] = ('pyproject.toml', '\n'.join(self.distribution.metadata.license_files))

(And then add that to cmdclass)

[abravalheri]

The original problem seems to be already solved with the latest versions of setuptools and wheel:

> docker run --rm -it python:3.11 /bin/bash


mkdir /tmp/test-licenses
cd /tmp/test-licenses
mkdir -p licenses
touch LICENSE
date > licenses/license-A.txt
cat > pyproject.toml << EOF
[build-system]
requires = ['setuptools==66.0.0', 'wheel==0.38.4']
build-backend = "setuptools.build_meta"
EOF
cat > setup.py <<EOF
from setuptools import setup

setup(
    name="repro-case",
    version='2.5.0dev0',
    python_requires="~=3.7",
    license="Apache License 2.0",
    author="Ash <[email protected]>",
    license_files=[
        'LICENSE',
        'licenses/license-A.txt',
    ],
)
EOF
rm -rf repro_case.egg-info/ dist
python -m pip install build
python -m build
unzip -l dist/*.whl
# Archive:  dist/repro_case-2.5.0.dev0-py3-none-any.whl
#   Length      Date    Time    Name
# ---------  ---------- -----   ----
#         0  2023-01-20 10:59   repro_case-2.5.0.dev0.dist-info/LICENSE
#       198  2023-01-20 10:59   repro_case-2.5.0.dev0.dist-info/METADATA
#        92  2023-01-20 10:59   repro_case-2.5.0.dev0.dist-info/WHEEL
#        29  2023-01-20 10:59   repro_case-2.5.0.dev0.dist-info/license-A.txt
#         9  2023-01-20 10:59   repro_case-2.5.0.dev0.dist-info/top_level.txt
#       521  2023-01-20 10:59   repro_case-2.5.0.dev0.dist-info/RECORD
# ---------                     -------
#       849                     6 files

Details about the exact placement of the license files inside .dist-info are likely to change once PEP 639 is implemented.