[new check] 'paths_exist'

[jameslamb]

What should be checked?

Checks that specific paths exist in the archive.

This is common in conda recipes. For example, like this from libpng's recipe.

test:
  commands:
    - test -f ${PREFIX}/lib/libpng.a  # [not win]
    - test -f ${PREFIX}/lib/libpng${SHLIB_EXT}  # [not win]

(code link)

That's convenient in conda packaging because conda build allows you to run arbitrary commands in the build environment before those contents are zipped up into an archive. That works well in conda builds because there's a well defined place where just the files that'll be in the package are staged (see the conda docs).

In building wheels and sdists, that's not typically true. You usually invoke tools like pip wheel, python -m build, poetry build, etc. in a source tree that contains a mix of files that'll end up in the package and those that don't, and then when that command exits, an archive has been created.

I think it'd be useful for pydistcheck to run such tests, especially for wheels and sdists. No more having to manually verify that your combination of MANIFEST.in, package_data, setup.py, and PEP-517 backend are doing what you expect 😁

What should the name of this check be?

expected-paths-not-found

Will this check introduce any additional configuration?

yes

Details on additional configuration required.

This should probably be empty and do nothing by default. pydistcheck is intended to not be too opinionated about the particular way that Python packages are laid out or even really to specifically couple too tightly to only checking strictly "python" packages.

From the command line, this should be supported via an option that can be specified multiple times, like this:

pydistcheck \
   --expect-path 'lib/whatever.so' \
   --expect-path 'LICENSE' \
   --expect-path '__init__.py'

In pyproject.toml, it should be something like this:

[tool.pydistcheck]
expect_path = [
    "lib/whatever.so",
    "LICENSE",
    "__init__.py"
]

I don't think this should support globbing (at least in the first implementation).

Distribution type

• source (e.g. .tar.gz, .zip)
• built (e.g. .conda, .tar.bz2, .whl)

Notes

Special consideration will need to be given for the .conda format, which is an archive that itself contains other specially-named archives. Like do those intermediate archive names need to be in the paths? (I think no)

If it's not going to support globs, then does this need to support some type of platform markers like conda does with e.g. # [win]? If not, what do you do if you want to use the same configuration but on different platforms, where e.g. you might have a lib/whatever.dll on Windows and lib/whatever.so on Linux?

Should this specific option from the command line cause the pyproject.toml configuration to be ignored (as all other configuration currently works)? Or should it be merged with whatever's in pyproject.toml?