Markdown README not displaying on PyPI (despite Description-Content-Type: text/markdown)

[bittner]

Environment

• OS/distro: Ubuntu 18.04.1 Bionic
• Python 3.7.0
• pip 18.0 (python 3.7)
• twine version 1.12.1 (pkginfo: 1.4.2, requests: 2.19.1, setuptools: 40.4.3, requests-toolbelt: 0.8.0, tqdm: 4.26.0)

Problem Description

When publishing a project with a README.md (Markdown) as the long_description and long_description_content_type='text/markdown' then PyPI displays all the submitted content in raw text format (instead of converted to HTML).

def read_file(filename):
    with open(os.path.join(os.path.dirname(__file__), filename)) as file:
        return file.read()

setup(
    name=package.name,
    version=package.__version__,
    description=package.__doc__.strip(),
    long_description=read_file('README.md'),
    long_description_content_type='text/markdown',
    ...

Expected Behavior

1. PyPI should display the README correctly (e.g. as on GitHub) on PyPI.
2. If there is a conversion problem the uploading tool should report an error, and abort; ideally, PyPI should refuse to accept content it won't display nicely.

Example Project

• https://test.pypi.org/manage/project/PythonTurtle/releases/

See also: PythonTurtle/PythonTurtle#121

[benoit-pierre]

Looks like a setuptools/wheel issue; as PyPI does work as expected in other cases (e.g.: https://pypi.org/project/black/ or https://test.pypi.org/project/black/). Looking at the metadata (PythonTurtle-0.3.3/PKG-INFO in the source distribution or PythonTurtle-0.3.3.dist-info/METADATA in the wheel), I can see that the description is not at the end, and you'll notice that PyPI does render the fields that follow as part of the description:

[...]
 PythonTurtle was created by Ram Rachum as a side-project in 2009. I also provide
[freelance Django/Python development services](https://chipmunkdev.com).
Keywords: turtle,learning,children,beginners,logo
Platform: UNKNOWN
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: MacOS X
Classifier: Environment :: Win32 (MS Windows)
Classifier: Environment :: X11 Applications :: GTK
Classifier: Intended Audience :: Education
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: MacOS
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Topic :: Education
Description-Content-Type: text/markdown

Among those field is Description-Content-Type, which would explain why its ignored and the description is rendered as reStructuredText.

[benoit-pierre]

You're getting hit by this issue: pypa/setuptools#1390.

[bittner]

It bug was: description must by all means be a single line. I'm now parsing it out of __doc__:

description=package.__doc__.strip().split('\n')[0]

Would be cool if there was a way to detect that as a problem and abort the upload.

[benoit-pierre]

You can't detect this at the upload level: it is valid, the problem must be detected at build time by setuptools.

[bittner]

Closing in favor of pypa/setuptools#1390 (duplicate).