bibtex entry on PyPI/TestPyPI renders strikethrough element <s> around curly braces

[matthewfeickert]

Describe the bug

This is scikit-hep/pyhf#2024.

On PyPI and TestPyPI the following RST

.. code:: bibtex

   @software{pyhf,
     author = {Lukas Heinrich and Matthew Feickert and Giordon Stark},
     title = "{pyhf: v0.7.0rc4}",
     version = {0.7.0rc4},
     doi = {10.5281/zenodo.1169739},
     url = {https://doi.org/10.5281/zenodo.1169739},
     note = {https://github.com/scikit-hep/pyhf/releases/tag/v0.7.0rc4}
   }

   @article{pyhf_joss,
     doi = {10.21105/joss.02823},
     url = {https://doi.org/10.21105/joss.02823},
     year = {2021},
     publisher = {The Open Journal},
     volume = {6},
     number = {58},
     pages = {2823},
     author = {Lukas Heinrich and Matthew Feickert and Giordon Stark and Kyle Cranmer},
     title = {pyhf: pure-Python implementation of HistFactory statistical models},
     journal = {Journal of Open Source Software}
   }

gets rendered out with the strikethrough element <s> around curly braces:

@software{pyhf,
  author = <s>{Lukas Heinrich and Matthew Feickert and Giordon Stark}</s>,
  title = <s>"{pyhf: v0.7.0rc4}"</s>,
  version = <s>{0.7.0rc4}</s>,
  doi = <s>{10.5281/zenodo.1169739}</s>,
  url = <s>{https://doi.org/10.5281/zenodo.1169739}</s>,
  note = <s>{https://github.com/scikit-hep/pyhf/releases/tag/v0.7.0rc4}</s>
}

@article{pyhf_joss,
  doi = <s>{10.21105/joss.02823}</s>,
  url = <s>{https://doi.org/10.21105/joss.02823}</s>,
  year = <s>{2021}</s>,
  publisher = <s>{The Open Journal}</s>,
  volume = <s>{6}</s>,
  number = <s>{58}</s>,
  pages = <s>{2823}</s>,
  author = <s>{Lukas Heinrich and Matthew Feickert and Giordon Stark and Kyle Cranmer}</s>,
  title = <s>{pyhf: pure-Python implementation of HistFactory statistical models}</s>,
  journal = <s>{Journal of Open Source Software}</s>
}

Expected behavior

I would expect that it get rendered out correctly without the <s> like it is in the project GitHub README and on the project docs.

To Reproduce

• On PyPI: https://pypi.org/project/pyhf/0.6.3/
• On TestPyPI: https://test.pypi.org/project/pyhf/0.7.0rc4/

My Platform

This is not platform dependent. I've tested this on Chrome and Firefox on both Linux and Mac.

Additional context

None.

[miketheman]

Thanks for the detailed example, I'll take a deeper look at this soon.

[miketheman]

This seems to stem from the setting in place to support Pygments highlighting:

readme_renderer/readme_renderer/rst.py

Lines 96 to 98 in e662bf4

	# Use the short form of syntax highlighting so that the generated
	# Pygments CSS can be used to style the output.
	"syntax_highlight": "short",

Removing this setting extracts the expected content. We use the Pygments short codes in warehouse CSS here: https://github.com/pypi/warehouse/blob/665eb50ce0066afa555f34c1ae5637bdd9361b59/warehouse/static/sass/blocks/_project-description.scss#L283-L516

It's unclear why docutils would behave differently for bibtex for this section only.

[miketheman]

Digging further, it's not docutils at all, it's driven from Pygments syntax highlighter, as it establishes a short name token for syntax classes, and has been s for years. https://github.com/pygments/pygments/blob/b58e855e41b9266f8412823dbb260d1b8e49b45f/pygments/token.py#L165

It looks like with the changeover to the HTML5 writer in #253 the <s> attribute wasn't allowed through our cleaner, and is thus escaped to a string value of <s>{2021}</s> instead of <s>{2021}</s> which warehouse CSS would pick up and handle as .project-description.s appropriately.

[miketheman]

@matthewfeickert The code here is fixed, and once warehouse pulls in the new version there's an automated process to re-render READMEs with the new code, so there's no further action needed on your end!

[matthewfeickert]

Fantastic. Thanks so much @miketheman!