Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FR] support pep-621 - storing core project metadata in pyproject.toml #2671

Closed
1 of 5 tasks
RonnyPfannschmidt opened this issue May 11, 2021 · 21 comments
Closed
1 of 5 tasks
Labels
enhancement major Needs Discussion Issues where the implementation still needs to be discussed. Needs Implementation Issues that are ready to be implemented.

Comments

@RonnyPfannschmidt
Copy link
Contributor

RonnyPfannschmidt commented May 11, 2021

What's the problem this feature will solve?

this is intended to build on #1688

with PEP-621 python packaging has been extended with a standard to include core metadata in pyproject.toml

Describe the solution you'd like

in order for setuptools to support it i beleive i see the need for the following steps

  • basic toml support in setuptools (just mirroring setup.cfg in pyproject.toml, as setup.cfg is still a supoerset of that the peps support
  • a loader for pep-621 meta-data (ideally via packaging)
  • [FR] Allow plugins to pull in configuration from pyproject.toml #3415
  • warnings for usages of tool.setuptools when core package metadata is viable (maybe via jsonschema + its deprecation mechanism)

Alternative Solutions

No response

Additional context

Support incremental complexity and build steps for packages

Code of Conduct

  • I agree to follow the PSF Code of Conduct
@RonnyPfannschmidt RonnyPfannschmidt added Needs Triage Issues that need to be evaluated for severity and status. enhancement labels May 11, 2021
@pganssle pganssle added major Needs Discussion Issues where the implementation still needs to be discussed. Needs Implementation Issues that are ready to be implemented. and removed Needs Triage Issues that need to be evaluated for severity and status. labels May 26, 2021
@abravalheri
Copy link
Contributor

abravalheri commented Oct 15, 2021

Hi @RonnyPfannschmidt, I was just wondering if you could clarify a bit more the requirement:

warnings for usages of tool.setuptools when core package metadata is viable (maybe via jsonschema + its deprecation mechanism)

I have been working on a tool to assist the migration once PEP621 is adopted by setuptools, but since not all information in setup.cfg is covered by the PEP, I kind of assumed a tool.setuptools table is required in the TOML file... (the assumption is only provisional, and subject to change once we have a clearer idea on how to encode those extra fields).

Does the requirement you wrote mean that tool.setuptools should not be used?

@RonnyPfannschmidt
Copy link
Contributor Author

@abravalheri if people put Metadata that has a location as per the pep into the tool. Setuptools table instead, there should be a warning /error

The table is absolutely needed as setup.cfg is a superset of the Metadata the pep directly in cooperates

@abravalheri
Copy link
Contributor

Thank you very much for the clarification @RonnyPfannschmidt.

abravalheri added a commit to abravalheri/setuptools that referenced this issue Dec 11, 2021
In order to minimise dependencies, `validate-pyproject` has the ability
to "dump" only the code necessary to run the validations to a given
directory. This special strategy is used instead of the default
`pip install -t`.

The idea of using JSONSchema for validation was suggested in pypa#2671,
and the rationale for that approach is further discussed in
https://github.com/abravalheri/validate-pyproject/blob/main/docs/faq.rst

Using a library such as `validate-pyproject` has the advantage of
incentive sing reuse and collaboration with other projects.

Currently `validate-pyproject` ships a JSONSchema for the proposed
use of `pyproject.toml` as means of configuration for setuptools.
In the future, if there is interest, setuptools could also ship its own
schema and just use the shared infrastructure of `validate-pyproject`
(by advertising the schemas via entry-points).
abravalheri added a commit to abravalheri/setuptools that referenced this issue Dec 20, 2021
In order to minimise dependencies, `validate-pyproject` has the ability
to "dump" only the code necessary to run the validations to a given
directory. This special strategy is used instead of the default
`pip install -t`.

The idea of using JSONSchema for validation was suggested in pypa#2671,
and the rationale for that approach is further discussed in
https://github.com/abravalheri/validate-pyproject/blob/main/docs/faq.rst

Using a library such as `validate-pyproject` has the advantage of
incentive sing reuse and collaboration with other projects.

Currently `validate-pyproject` ships a JSONSchema for the proposed
use of `pyproject.toml` as means of configuration for setuptools.
In the future, if there is interest, setuptools could also ship its own
schema and just use the shared infrastructure of `validate-pyproject`
(by advertising the schemas via entry-points).
abravalheri added a commit to abravalheri/setuptools that referenced this issue Dec 23, 2021
In order to minimise dependencies, `validate-pyproject` has the ability
to "dump" only the code necessary to run the validations to a given
directory. This special strategy is used instead of the default
`pip install -t`.

The idea of using JSONSchema for validation was suggested in pypa#2671,
and the rationale for that approach is further discussed in
https://github.com/abravalheri/validate-pyproject/blob/main/docs/faq.rst

Using a library such as `validate-pyproject` has the advantage of
incentive sing reuse and collaboration with other projects.

Currently `validate-pyproject` ships a JSONSchema for the proposed
use of `pyproject.toml` as means of configuration for setuptools.
In the future, if there is interest, setuptools could also ship its own
schema and just use the shared infrastructure of `validate-pyproject`
(by advertising the schemas via entry-points).
abravalheri added a commit to abravalheri/setuptools that referenced this issue Dec 25, 2021
In order to minimise dependencies, `validate-pyproject` has the ability
to "dump" only the code necessary to run the validations to a given
directory. This special strategy is used instead of the default
`pip install -t`.

The idea of using JSONSchema for validation was suggested in pypa#2671,
and the rationale for that approach is further discussed in
https://github.com/abravalheri/validate-pyproject/blob/main/docs/faq.rst

Using a library such as `validate-pyproject` has the advantage of
incentive sing reuse and collaboration with other projects.

Currently `validate-pyproject` ships a JSONSchema for the proposed
use of `pyproject.toml` as means of configuration for setuptools.
In the future, if there is interest, setuptools could also ship its own
schema and just use the shared infrastructure of `validate-pyproject`
(by advertising the schemas via entry-points).
abravalheri added a commit to abravalheri/setuptools that referenced this issue Dec 25, 2021
In order to minimise dependencies, `validate-pyproject` has the ability
to "dump" only the code necessary to run the validations to a given
directory. This special strategy is used instead of the default
`pip install -t`.

The idea of using JSONSchema for validation was suggested in pypa#2671,
and the rationale for that approach is further discussed in
https://github.com/abravalheri/validate-pyproject/blob/main/docs/faq.rst

Using a library such as `validate-pyproject` has the advantage of
incentive sing reuse and collaboration with other projects.

Currently `validate-pyproject` ships a JSONSchema for the proposed
use of `pyproject.toml` as means of configuration for setuptools.
In the future, if there is interest, setuptools could also ship its own
schema and just use the shared infrastructure of `validate-pyproject`
(by advertising the schemas via entry-points).
abravalheri added a commit to abravalheri/setuptools that referenced this issue Jan 7, 2022
In order to minimise dependencies, `validate-pyproject` has the ability
to "dump" only the code necessary to run the validations to a given
directory. This special strategy is used instead of the default
`pip install -t`.

The idea of using JSONSchema for validation was suggested in pypa#2671,
and the rationale for that approach is further discussed in
https://github.com/abravalheri/validate-pyproject/blob/main/docs/faq.rst

Using a library such as `validate-pyproject` has the advantage of
incentive sing reuse and collaboration with other projects.

Currently `validate-pyproject` ships a JSONSchema for the proposed
use of `pyproject.toml` as means of configuration for setuptools.
In the future, if there is interest, setuptools could also ship its own
schema and just use the shared infrastructure of `validate-pyproject`
(by advertising the schemas via entry-points).
abravalheri added a commit to abravalheri/setuptools that referenced this issue Jan 24, 2022
In order to minimise dependencies, `validate-pyproject` has the ability
to "dump" only the code necessary to run the validations to a given
directory. This special strategy is used instead of the default
`pip install -t`.

The idea of using JSONSchema for validation was suggested in pypa#2671,
and the rationale for that approach is further discussed in
https://github.com/abravalheri/validate-pyproject/blob/main/docs/faq.rst

Using a library such as `validate-pyproject` has the advantage of
incentive sing reuse and collaboration with other projects.

Currently `validate-pyproject` ships a JSONSchema for the proposed
use of `pyproject.toml` as means of configuration for setuptools.
In the future, if there is interest, setuptools could also ship its own
schema and just use the shared infrastructure of `validate-pyproject`
(by advertising the schemas via entry-points).
abravalheri added a commit to abravalheri/setuptools that referenced this issue Feb 1, 2022
In order to minimise dependencies, `validate-pyproject` has the ability
to "dump" only the code necessary to run the validations to a given
directory. This special strategy is used instead of the default
`pip install -t`.

The idea of using JSONSchema for validation was suggested in pypa#2671,
and the rationale for that approach is further discussed in
https://github.com/abravalheri/validate-pyproject/blob/main/docs/faq.rst

Using a library such as `validate-pyproject` has the advantage of
incentive sing reuse and collaboration with other projects.

Currently `validate-pyproject` ships a JSONSchema for the proposed
use of `pyproject.toml` as means of configuration for setuptools.
In the future, if there is interest, setuptools could also ship its own
schema and just use the shared infrastructure of `validate-pyproject`
(by advertising the schemas via entry-points).
abravalheri added a commit that referenced this issue Feb 1, 2022
In order to minimise dependencies, `validate-pyproject` has the ability
to "dump" only the code necessary to run the validations to a given
directory. This special strategy is used instead of the default
`pip install -t`.

The idea of using JSONSchema for validation was suggested in #2671,
and the rationale for that approach is further discussed in
https://github.com/abravalheri/validate-pyproject/blob/main/docs/faq.rst

Using a library such as `validate-pyproject` has the advantage of
incentive sing reuse and collaboration with other projects.

Currently `validate-pyproject` ships a JSONSchema for the proposed
use of `pyproject.toml` as means of configuration for setuptools.
In the future, if there is interest, setuptools could also ship its own
schema and just use the shared infrastructure of `validate-pyproject`
(by advertising the schemas via entry-points).
abravalheri added a commit to abravalheri/setuptools that referenced this issue Feb 7, 2022
In order to minimise dependencies, `validate-pyproject` has the ability
to "dump" only the code necessary to run the validations to a given
directory. This special strategy is used instead of the default
`pip install -t`.

The idea of using JSONSchema for validation was suggested in pypa#2671,
and the rationale for that approach is further discussed in
https://github.com/abravalheri/validate-pyproject/blob/main/docs/faq.rst

Using a library such as `validate-pyproject` has the advantage of
incentive sing reuse and collaboration with other projects.

Currently `validate-pyproject` ships a JSONSchema for the proposed
use of `pyproject.toml` as means of configuration for setuptools.
In the future, if there is interest, setuptools could also ship its own
schema and just use the shared infrastructure of `validate-pyproject`
(by advertising the schemas via entry-points).
abravalheri added a commit that referenced this issue Feb 7, 2022
In order to minimise dependencies, `validate-pyproject` has the ability
to "dump" only the code necessary to run the validations to a given
directory. This special strategy is used instead of the default
`pip install -t`.

The idea of using JSONSchema for validation was suggested in #2671,
and the rationale for that approach is further discussed in
https://github.com/abravalheri/validate-pyproject/blob/main/docs/faq.rst

Using a library such as `validate-pyproject` has the advantage of
incentive sing reuse and collaboration with other projects.

Currently `validate-pyproject` ships a JSONSchema for the proposed
use of `pyproject.toml` as means of configuration for setuptools.
In the future, if there is interest, setuptools could also ship its own
schema and just use the shared infrastructure of `validate-pyproject`
(by advertising the schemas via entry-points).
abravalheri added a commit that referenced this issue Feb 9, 2022
In order to minimise dependencies, `validate-pyproject` has the ability
to "dump" only the code necessary to run the validations to a given
directory. This special strategy is used instead of the default
`pip install -t`.

The idea of using JSONSchema for validation was suggested in #2671,
and the rationale for that approach is further discussed in
https://github.com/abravalheri/validate-pyproject/blob/main/docs/faq.rst

Using a library such as `validate-pyproject` has the advantage of
incentive sing reuse and collaboration with other projects.

Currently `validate-pyproject` ships a JSONSchema for the proposed
use of `pyproject.toml` as means of configuration for setuptools.
In the future, if there is interest, setuptools could also ship its own
schema and just use the shared infrastructure of `validate-pyproject`
(by advertising the schemas via entry-points).
abravalheri added a commit that referenced this issue Feb 11, 2022
In order to minimise dependencies, `validate-pyproject` has the ability
to "dump" only the code necessary to run the validations to a given
directory. This special strategy is used instead of the default
`pip install -t`.

The idea of using JSONSchema for validation was suggested in #2671,
and the rationale for that approach is further discussed in
https://github.com/abravalheri/validate-pyproject/blob/main/docs/faq.rst

Using a library such as `validate-pyproject` has the advantage of
incentive sing reuse and collaboration with other projects.

Currently `validate-pyproject` ships a JSONSchema for the proposed
use of `pyproject.toml` as means of configuration for setuptools.
In the future, if there is interest, setuptools could also ship its own
schema and just use the shared infrastructure of `validate-pyproject`
(by advertising the schemas via entry-points).
@con-f-use
Copy link
Contributor

con-f-use commented Feb 12, 2022

Just for clarification, because true to my nick, I'm a little bit confused. Can I put core metadata from PEP 621 into my pyproject.toml and expect that work with setuptools? What happens today (version 60.8.2) if I put core metadata into pyproject.toml and delete the duplicates of these metadata keys from setup.cfg?

@abravalheri
Copy link
Contributor

Hi @con-f-use, not yet. This feature is still under active development/review.

abravalheri added a commit that referenced this issue Feb 18, 2022
In order to minimise dependencies, `validate-pyproject` has the ability
to "dump" only the code necessary to run the validations to a given
directory. This special strategy is used instead of the default
`pip install -t`.

The idea of using JSONSchema for validation was suggested in #2671,
and the rationale for that approach is further discussed in
https://github.com/abravalheri/validate-pyproject/blob/main/docs/faq.rst

Using a library such as `validate-pyproject` has the advantage of
incentive sing reuse and collaboration with other projects.

Currently `validate-pyproject` ships a JSONSchema for the proposed
use of `pyproject.toml` as means of configuration for setuptools.
In the future, if there is interest, setuptools could also ship its own
schema and just use the shared infrastructure of `validate-pyproject`
(by advertising the schemas via entry-points).
abravalheri added a commit that referenced this issue Feb 19, 2022
In order to minimise dependencies, `validate-pyproject` has the ability
to "dump" only the code necessary to run the validations to a given
directory. This special strategy is used instead of the default
`pip install -t`.

The idea of using JSONSchema for validation was suggested in #2671,
and the rationale for that approach is further discussed in
https://github.com/abravalheri/validate-pyproject/blob/main/docs/faq.rst

Using a library such as `validate-pyproject` has the advantage of
incentive sing reuse and collaboration with other projects.

Currently `validate-pyproject` ships a JSONSchema for the proposed
use of `pyproject.toml` as means of configuration for setuptools.
In the future, if there is interest, setuptools could also ship its own
schema and just use the shared infrastructure of `validate-pyproject`
(by advertising the schemas via entry-points).
abravalheri added a commit that referenced this issue Mar 5, 2022
In order to minimise dependencies, `validate-pyproject` has the ability
to "dump" only the code necessary to run the validations to a given
directory. This special strategy is used instead of the default
`pip install -t`.

The idea of using JSONSchema for validation was suggested in #2671,
and the rationale for that approach is further discussed in
https://github.com/abravalheri/validate-pyproject/blob/main/docs/faq.rst

Using a library such as `validate-pyproject` has the advantage of
incentive sing reuse and collaboration with other projects.

Currently `validate-pyproject` ships a JSONSchema for the proposed
use of `pyproject.toml` as means of configuration for setuptools.
In the future, if there is interest, setuptools could also ship its own
schema and just use the shared infrastructure of `validate-pyproject`
(by advertising the schemas via entry-points).
@abravalheri
Copy link
Contributor

Hi @RonnyPfannschmidt, the basics of the support have been implemented now.
Do you think there is something left in this ticket that is not implemented yet? Or should we just close it?

@RonnyPfannschmidt
Copy link
Contributor Author

@abravalheri i haven't been on top of this one

Is toml Metadata avaliable to setuptools extensions?

@abravalheri
Copy link
Contributor

Could you describe a bit more what making toml Metadata avaliable to extensions would mean?
Is there any existing mechanism for setup.cfg metadata?

@RonnyPfannschmidt
Copy link
Contributor Author

No, which makes Integration painful

@abravalheri
Copy link
Contributor

Thank you very much Ronny for clarifying that.
This looks like a fair request. If we manage to put up the requirements for this feature (or at least have an idea on how it would look like), I can do some investigation and work on that sometime in the near future (no promises though :P).

@abravalheri
Copy link
Contributor

Depending on the hook the plugin implements, the dist object will have its attributes set to the corresponding values in pyproject.toml, the problem is that the majority of the existing hooks run before the parsing of the configuration.

Are the dist attributes enough or would this feature be something different?

@bhrutledge
Copy link
Contributor

bhrutledge commented May 31, 2022

@abravalheri @RonnyPfannschmidt If the basics of PEP 621 support have been implemented (per #2671 (comment)), can further enhancements be submitted as separate issues? I think that would allow this issue to be closed, making the development status clearer to other folks.

I note that there's a checklist in the original description; what of that has been implemented, and what could be a separate issue?

Also, the setuptools docs still say that support is experimental. I think it'd be helpful for that document to include or link to a list of what's stable, and what's in progress.

Maybe setting up a project for PEP 621 support would be helpful?

@abravalheri
Copy link
Contributor

Thank you very much @bhrutledge, If Ronny is OK with that I think we could split the remaining of the feature request in another issue.

Regarding the experimental part, after these few months that PEP 621 support was implemented in setuptools, I believe that we reached some level of stability, and I don't see this feature being removed.

The experimental part is the [tool.setuptools] table which is not covered by the PEP and contains other levers and knobs specific for setuptools. This other table is not mandatory though, and simple use cases should work perfectly fine without it.

I need to update the docs to make this clear.

@bhrutledge
Copy link
Contributor

Thanks @abravalheri! I believe there's also a warning in the output of python3 -m build that says pyproject.toml support is experimental.

@abravalheri
Copy link
Contributor

Thank you @bhrutledge, I will have a look on that.

@abravalheri
Copy link
Contributor

I have started working on #3347 to address the points brought up by @bhrutledge. Hopefully it reflects better the status of the feature.

@RonnyPfannschmidt
Copy link
Contributor Author

👍

@abravalheri
Copy link
Contributor

abravalheri commented Jun 26, 2022

I am splitting the remaining aspects of the feature request in #3415.
Anyone interested please feel free to add comments there.

@agronholm
Copy link
Contributor

Did you just link to this same issue?

@abravalheri
Copy link
Contributor

abravalheri commented Jun 26, 2022

Woops, thanks @agronholm. The correct issue is #3415.

@abravalheri
Copy link
Contributor

My interpretation of the status of the original steps is:

  • basic toml support in setuptools (just mirroring setup.cfg in pyproject.toml, as setup.cfg is still a supoerset of that the peps support

    Basic support for pyproject.toml configuration is currently implemented in setuptools.
    I am being very conservative about the [tool.setuptools] table because I want us to use this opportunity to reflect what would be the best shape for this data structure (ideally simplifying stuff), not only mirror what is currently available in setup.cfg.
    I cannot do that alone, though, so I would appreciate input on this aspect.

  • a loader for pep-621 meta-data (ideally via packaging)

    The existing implementation uses tomli to load pyproject.toml and then a series of custom functions to apply the data to the distribution class. But I am happy to replace it if packaging implements the required APIs. Related to the discussion of A metadata module with a data class for core metadata packaging#518.

  • hooks to pull in the pep-621 metadata

    I understand that this refers to setuptools plugins. I am tracking this in [FR] Allow plugins to pull in configuration from pyproject.toml #3415

  • warnings for usages of tool.setuptools when core package metadata is viable (maybe via jsonschema + its deprecation mechanism)

    The existing implementation will raise a validation error if someone tries to specify tool.setuptools.name, tool.setuptools.version etc...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement major Needs Discussion Issues where the implementation still needs to be discussed. Needs Implementation Issues that are ready to be implemented.
Projects
None yet
Development

No branches or pull requests

6 participants