Add a lint check to ensure classifiers (and other metadata values) are valid

[pfmoore]

Pip 21.0 has now been released.

One issue, around the classifiers (a previous commit had left them incorrect). We should probably have a lint check that the classifiers are valid, to avoid this sort of issue in the future.

Originally posted by @pfmoore in #9282 (comment)

[uranusjr]

Is there a tool for this?

[pradyunsg]

Hmm... Maybe @di would know.

[hugovk]

I don't know if there's a tool, but it's planned for twine check: see pypa/twine#430 and
pypa/packaging#147

---

In the meantime, Trove classifiers can be checked using https://pypi.org/project/trove-classifiers/

Something along the lines of:

from trove_classifiers import classifiers  # pip install trove-classifiers

BAD_CLASSIFIERS = {
    "Development Status :: 5 - Production/Stable",
    "Intended Audience :: Developers",
    "License :: OSI Approved :: MIT License",
    "Topic :: Software Development :: Build Tools",
    "Programming Language :: Python",
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3 :: Only"  # missing comma!
    "Programming Language :: Python :: 3.6",
    "Programming Language :: Python :: 3.7",
    "Programming Language :: Python :: 3.8",
    "Programming Language :: Python :: 3.9",
    "Programming Language :: Python :: Implementation :: CPython",
    "Programming Language :: Python :: Implementation :: PyPy",
}

GOOD_CLASSIFIERS = {
    "Development Status :: 5 - Production/Stable",
    "Intended Audience :: Developers",
    "License :: OSI Approved :: MIT License",
    "Topic :: Software Development :: Build Tools",
    "Programming Language :: Python",
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3 :: Only",  # fixed comma
    "Programming Language :: Python :: 3.6",
    "Programming Language :: Python :: 3.7",
    "Programming Language :: Python :: 3.8",
    "Programming Language :: Python :: 3.9",
    "Programming Language :: Python :: Implementation :: CPython",
    "Programming Language :: Python :: Implementation :: PyPy",
}


print(BAD_CLASSIFIERS <= classifiers)  # False

print(GOOD_CLASSIFIERS <= classifiers)  # True

---

Alternatively, python -m trove_classifiers lists the valid ones, and python setup.py --classifiers lists ours.

For example, using some Unix subset piping:

$ python -m trove_classifiers > all.txt
$ python setup.py --classifiers > ours.txt
$ comm -23 <(sort -n ours.txt|uniq) <(sort -n all.txt|uniq) | head -1
Programming Language :: Python :: 3 :: OnlyProgramming Language :: Python :: 3.6
$ # Now fix setup.py
$ edit setup.py
$ python setup.py --classifiers > ours.txt
$ comm -23 <(sort -n ours.txt|uniq) <(sort -n all.txt|uniq) | head -1
$

And put a test to ensure the subset output is empty.

(Could probably ditch the sort and uniq too.)

[uranusjr]

It should not be too difficult to write a pre-commit hook wrapping trove-classifiers. I wonder if someone has done it already.

[di]

Yes, eventually twine --check should do what you want here. Probably worth including that in your CI anyways (right now it just verifies that a long_description will render.

[pradyunsg]

We do run twine check in our artifact build pipeline, and that is run in the CI as well.

pip/noxfile.py

Line 312 in e17ddea

session.run("twine", "check", *produced_dists, silent=True)

pip/.azure-pipelines/jobs/package.yml

Line 29 in e17ddea

- bash: nox -s build-release -- 99.9