Create draft PEP

[pombredanne]

The goal is to add support for SPDX licenses ID in Pypi packages.
Based on a discussion started by @devcurmudgeon and @jaraco in jaraco/keyring#263

@goneall @sschuberth @hugo-bmwcarit you may be interested by this too
If so tell it here and I will add you to the repo.

[pombredanne]

Here is a first batch of ideas and issues... not well organized yet.

Context

Software is licensed and providing accurate licensing information to Python packages users is an important (or essential) matter.

Problem

The licensing of Python packages is under-documented in many cases leading to some head scratching or outright immediate non-compliance for users of a package. Clarity in licensing is rather useful to a user.

There are various conflicting and overlapping places where license can be documented leading to confusion for packagers. Documenting licensing is under-specified leading to inconsistencies when licensing is documented. Clarity when creating packages with one way to do things is rather useful for packagers.

Ideal Solution

As a Python packager, I want to have a single and clear way to document the licensing of my package both at a summary and detailed level such that it is unambiguous and clearly accessible to my users.
This licensing should be validated as consistent and I should document this in as few files as possible and without redundant entries.

As a Python package user, I want to get clear licensing information both at a summary and detailed level when:

• I browse Pypi
• I inspect a package archive
• I inspect an installed package

Now what are the (many) issues that this PEP could address (or to be split in other PEPs)?

1. Add SPDX license ids to the list of Pypi classifiers at https://pypi.python.org/pypi?:action=list_classifiers

• This may not require a PEP and could possibly be just a patch to submit

• Note that the Pypi licensing classifiers are not perfect as they are, and several list outdated, exotic licenses and or licensing that cannot be resolved to a clear license : these are base on OSI licenses and they are clearly dated.

• The relationship between these classifiers and the license arg is documented but not enforced. (see below). Note that the classifiers are enforced for correctness when uploading to Pypi: a package cannot be uploaded with incorrect classifiers.

• SPDX ids have been adopted by NPMs and Ruby FWIW.

2. Define a way to document more accurately the licensing of a package as metadata.

• Since classifiers are a fixed list it would not be possible to have a variable license expression such as: MIT or GPL-2.0 with Classpath-2.0. These are rare but classifiers would not allow any expressiveness of combination of licenses.

• setup() has a license argument which is free text today. It could be evolved such that it could be validated against a list of license ids or names and eventually validated as an expression composing these ids/names. The validation could be soft (e.g. trigger a warning display rather than an error) to avoid breaking everything. Per the Python docs this field is related to the classifier arg. Actual usage in practice is all over the map.

The license field is a text indicating the license covering the package where the license is not a selection from the “License” Trove classifiers. See the Classifier field. Notice that there’s a licence distribution option which is deprecated but still acts as an alias for license.

• There should be a way to point to one or more license and notice text files that document the license of a package. This could be part of the metadata in a setup() declaration or some other mechanism (setup.cfg ???, TBD)

• as side notes:

  • the CPython interpreter has built-in license() and copyright() functions that prints the Python licenses and copyright.
  • Python developers sometimes use these globals to document the license and copyright of their code such as __license__ = "BSD or Apache License, Version 2.0" and __copyright__ = "Copyright (c) J. Doe"

3. Ensure that defined licensing metadata and files are always included in built packages.

• metadata should end up in well defined places in built and installed packages and be accessible. pip -show` does it alright for wheels.

• Today downstream users are systematically not compliant with the license terms when built packages do not contain the license texts proper which is an essential condition of most licenses (such as the MIT and Python licenses themselves.)

  • sdist can include licenses texts with a MANIFEST.in but do not do it automatically

  • wheels have very limited provisions to include extra license texts (e.g. a single text file in the METADATA using a setup.cfg entry and do not honor MANIFEST.in entries ... See https://bitbucket.org/pypa/wheel/issues/138/include-more-than-one-license-file-or ... This eventually makes a wheel problematic even if the sdist contains the licensing alright.

So we could spec better where the licensing files and metadata end up in the built and installed packages. And ensure that they can be obtained easily (including possibly updating pip to deal with licensing files)

[kpfleming]

I fully support this proposal and would be happy to help draft the PEP and submit it for consideration.

[pombredanne]

@kpfleming Hey! thank you ++

[pombredanne]

@taleinat based on comments, we need a PEP after all, so let's start crafting one here.
Let me put down together some draft text here

[pombredanne]

@kpfleming @taleinat I gave you both push access to this repo.
And I started a draft PEP at python#1148

[pombredanne]

@tieguy ping too :)

[pombredanne]

And also https://discuss.python.org/t/improving-license-clarity-with-better-package-metadata/2154

[pombredanne]

@kpfleming @taleinat this is the draft PEP file https://github.com/pombredanne/spdx-pypi-pep/blob/metadata-2.2-spdx-license/pep-9999.rst ... You can edit directly in that branch (metadata-2.2-spdx-license) to update the draft (and the PR). For a start you can add yourself as co-authors there.
@Steap do you want to join to co-author this PEP?

Note: @pfmoore kindly offered to be both the BDFL delegate and the sponsor for this PEP 🙇‍♂️

[Steap]

@pombredanne I would love to join!

Shouldn't we keep the discussion going at https://discuss.python.org/t/improving-license-clarity-with-better-package-metadata/2154/15 for now?

[pombredanne]

@Steap I am giving you push access to this repo

Shouldn't we keep the discussion going at https://discuss.python.org/t/improving-license-clarity-with-better-package-metadata/2154/15 for now?

The process should be IMHO:

1. the discussion is on https://discuss.python.org/t/improving-license-clarity-with-better-package-metadata/2154/15
2. the results of the discussion end up in the PEP and PR (and the end results may not be a PEP in the end, but having a proper consolidated document with all in one place is better IMHO)

[pombredanne]

I am closing this as we have the PR and the discuss paces for feedback and review