Spack Create Python: Parse pyproject.toml?

[ax3l]

Wouldn't it be cool if we parsed the requirements.txt pyproject.toml of a python project on spack create to auto-populate the depends_on(..., 'build') packages we propose? :-)

[alalazo]

This is smaller in scope, but I think it's somehow related to #1249

[adamjstewart]

This is a bad idea. requirements.txt may contain some of the required dependencies, but it generally also includes several other dependencies that aren't necessary to build the project. These include dependencies required for unit testing and building the documentation.

[ax3l]

Ok, good to know. I saw several projects adding an extra requirements.txt for docs, testing and tutorials but if people commonly throw it all in there, which will probably also push their pypi deps up, then it might better be avoided.

Anyway, it's just a template proposal we would create from it, of course on can remove non-dependencies again :)

[ax3l]

Modernization of the issue: auto-populate the depends_on(..., 'build') lines with entries from pyproject.toml (PEP-517/518).
Example: https://github.com/openPMD/openPMD-api/blob/dev/pyproject.toml

[adamjstewart]

I don't know how common pyproject.toml is, but setup.cfg is becoming more popular and can also be parsed to locate dependency information in some cases.

[jcftang]

some of the more modern python tooling requires the pyproject.toml config file to be used, from experience enabling the use of the pyproject.toml file enforces pep517 which tends to break things in unexpected ways.

[ax3l]

The usage of both setup.cfg or pyproject.toml is a project choice and not up to us to add/judge ;)
PEP517 mainly breaks things because pip was not fully ready when it was rolled out and has quite some corner-cases that caused regressions in pip tooling.

[takluyver]

At some point Spack will probably need to support a new backend (or adapt PythonPackage - I don't know what's easier). pyproject.toml allows projects to skip having setup.py and be built with a standard interface defined by PEP 517.

[adamjstewart]

This is related to the discussion in #30795. Basically, we either try to parse setup.cfg, pyproject.toml, and requirements.txt (which may be nested, duplicated, or contain optional developer deps), or we parse the wheels directly. Just want to say that this is still on my radar and something I'm very interested in supporting.