Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Double underscores in .rst confuse parsing, new in 3.6 #10497

Closed
nedbat opened this issue Dec 26, 2024 · 2 comments
Closed

Double underscores in .rst confuse parsing, new in 3.6 #10497

nedbat opened this issue Dec 26, 2024 · 2 comments
Labels

Comments

@nedbat
Copy link

nedbat commented Dec 26, 2024

Whittled down from a much larger file: changes.rst is:

*  Added parameters to coverage.__init__ for options that had been set
   on the coverage object itself.

3.5 handled this correctly. 3.6 and 3.6.1 do not:

% ./pandoc3.5 -frst -tmarkdown_strict --markdown-headings=atx --wrap=none changes.rst
-   Added parameters to coverage.\_\_init\_\_ for options that had been set on the coverage object itself.

% ./pandoc3.6 -frst -tmarkdown_strict --markdown-headings=atx --wrap=none changes.rst
Error at "changes.rst_chunk" (line 2, column 1):
unexpected 'o'
expecting block

% ./pandoc3.6.1 -frst -tmarkdown_strict --markdown-headings=atx --wrap=none changes.rst
Error at "changes.rst_chunk" (line 2, column 1):
unexpected 'o'
expecting block

Fiddling with the underscores changes the behavior:

*  Added parameters to coverage.init__ for options that had been set
   on the coverage object itself.

Produces:

Error at "changes.rst_chunk" (line 4, column 1):
unexpected end of input
expecting end of input

and

*  Added parameters to coverage.init for options that had been set
   on the coverage object itself.

converts correctly.

@nedbat nedbat added the bug label Dec 26, 2024
@jgm
Copy link
Owner

jgm commented Dec 26, 2024

Even simpler case:

a.__b__

This should just be a literal a.__b__ but pandoc fails with

Error at (line 2, column 1):
unexpected '\n'
expecting block

Here is the list of RST reader changes in 3.6:

  * RST reader:

    + Use a new one-pass parsing strategy. Instead of having an initial pass
      where we collect reference definitions, we create links with target
      `##SUBST##something` or `##REF##something` or `##NOTE##something`,
      and resolve these in a pass over the parsed AST. This allows us to handle
      link references that are not at the top level (#10281).
    + Ignore newlines in URL in explicit link (#10279).
    + Handle block level substitutions.
    + Support `:file:` on raw directive (#8584).
    + Implement option lists (#10318).
    + Avoid putting metadata in Para (#7766). Create MetaInlines when possible,
      just as with markdown input. MetaBlocks is still used when there are
      multiple paragraphs or non-paragraph content.
      This change also affects field lists.
    + Fix linked substitutions (#6588). E.g. `|Python|_`.
    + Support inline anchors (#9196).
    + Explicit links define references (#5081). For example,
      ``Go to `g`_ `g <www.example.com>`_.`` should produce two links to
      www.example.com.

@jgm
Copy link
Owner

jgm commented Dec 26, 2024

Also:

% pandoc -f rst
a__b__
Error at (line 3, column 1):
unexpected end of input
expecting end of input

And

% pandoc -f rst
__b__
^D

(Succeeds but with no output.)

@jgm jgm closed this as completed in 9fa4fa3 Dec 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants