Add support for rowspan/colspan to tables

[utzig]

Better descriptions where added to the commits themselves. After this is merged, the only remaining support that needs to be added to tables is align. It also fixes the issues previously seen with the latex writer.

[vermeeren]

@utzig Changes so far look good to me, but I don't observe any changes on the table example page and make pdf does throw an exception. Presumably still work in progress?

[utzig]

@utzig Changes so far look good to me, but I don't observe any changes on the table example page and make pdf does throw an exception. Presumably still work in progress?

Not WIP, all the tables should render correctly with this PR, they do here, and I don't see issues with "make pdf" locally. Are you sure you tested with the PR's commits?

[vermeeren]

Not WIP, all the tables should render correctly with this PR, they do here, and I don't see issues with "make pdf" locally. Are you sure you tested with the PR's commits?

@utzig commit 451ad15, Sphinx 3.0.2, make pdf throws:

writing... failed
Exception occurred while building, starting debugger:
Traceback (most recent call last):
  File "/home/melvin/.local/lib/python3.7/site-packages/sphinx/cmd/build.py", line 280, in build_main
    app.build(args.force_all, filenames)
  File "/home/melvin/.local/lib/python3.7/site-packages/sphinx/application.py", line 348, in build
    self.builder.build_update()
  File "/home/melvin/.local/lib/python3.7/site-packages/sphinx/builders/__init__.py", line 294, in build_update
    self.build(['__all__'], to_build)
  File "/home/melvin/.local/lib/python3.7/site-packages/sphinx/builders/__init__.py", line 361, in build
    self.write(docnames, list(updated_docnames), method)
  File "/home/melvin/.local/lib/python3.7/site-packages/sphinx/builders/latex/__init__.py", line 301, in write
    docwriter.write(doctree, destination)
  File "/home/melvin/.local/lib/python3.7/site-packages/docutils/writers/__init__.py", line 78, in write
    self.translate()
  File "/home/melvin/.local/lib/python3.7/site-packages/sphinx/writers/latex.py", line 102, in translate
    self.document.walkabout(visitor)
  File "/home/melvin/.local/lib/python3.7/site-packages/docutils/nodes.py", line 214, in walkabout
    if child.walkabout(visitor):
  File "/home/melvin/.local/lib/python3.7/site-packages/docutils/nodes.py", line 214, in walkabout
    if child.walkabout(visitor):
  File "/home/melvin/.local/lib/python3.7/site-packages/docutils/nodes.py", line 214, in walkabout
    if child.walkabout(visitor):
  [Previous line repeated 9 more times]
  File "/home/melvin/.local/lib/python3.7/site-packages/docutils/nodes.py", line 227, in walkabout
    visitor.dispatch_departure(self)
  File "/home/melvin/.local/lib/python3.7/site-packages/sphinx/util/docutils.py", line 485, in dispatch_departure
    method(node)
  File "/home/melvin/.local/lib/python3.7/site-packages/sphinx/writers/latex.py", line 947, in depart_row
    underlined = [cell.row + cell.height == self.table.row + 1 for cell in cells]
  File "/home/melvin/.local/lib/python3.7/site-packages/sphinx/writers/latex.py", line 947, in <listcomp>
    underlined = [cell.row + cell.height == self.table.row + 1 for cell in cells]
AttributeError: 'NoneType' object has no attribute 'row'
> /home/melvin/.local/lib/python3.7/site-packages/sphinx/writers/latex.py(947)<listcomp>()
-> underlined = [cell.row + cell.height == self.table.row + 1 for cell in cells]
(Pdb)

Tables render the same as https://breathe.readthedocs.io/en/latest/tables.html with make html.

What Sphinx version do you have locally? I am thinking my ancient version is to blame here.

[utzig]

Not WIP, all the tables should render correctly with this PR, they do here, and I don't see issues with "make pdf" locally. Are you sure you tested with the PR's commits?

@utzig commit 451ad15, Sphinx 3.0.2, make pdf throws:

writing... failed
Exception occurred while building, starting debugger:
Traceback (most recent call last):
  File "/home/melvin/.local/lib/python3.7/site-packages/sphinx/cmd/build.py", line 280, in build_main
    app.build(args.force_all, filenames)
  File "/home/melvin/.local/lib/python3.7/site-packages/sphinx/application.py", line 348, in build
    self.builder.build_update()
  File "/home/melvin/.local/lib/python3.7/site-packages/sphinx/builders/__init__.py", line 294, in build_update
    self.build(['__all__'], to_build)
  File "/home/melvin/.local/lib/python3.7/site-packages/sphinx/builders/__init__.py", line 361, in build
    self.write(docnames, list(updated_docnames), method)
  File "/home/melvin/.local/lib/python3.7/site-packages/sphinx/builders/latex/__init__.py", line 301, in write
    docwriter.write(doctree, destination)
  File "/home/melvin/.local/lib/python3.7/site-packages/docutils/writers/__init__.py", line 78, in write
    self.translate()
  File "/home/melvin/.local/lib/python3.7/site-packages/sphinx/writers/latex.py", line 102, in translate
    self.document.walkabout(visitor)
  File "/home/melvin/.local/lib/python3.7/site-packages/docutils/nodes.py", line 214, in walkabout
    if child.walkabout(visitor):
  File "/home/melvin/.local/lib/python3.7/site-packages/docutils/nodes.py", line 214, in walkabout
    if child.walkabout(visitor):
  File "/home/melvin/.local/lib/python3.7/site-packages/docutils/nodes.py", line 214, in walkabout
    if child.walkabout(visitor):
  [Previous line repeated 9 more times]
  File "/home/melvin/.local/lib/python3.7/site-packages/docutils/nodes.py", line 227, in walkabout
    visitor.dispatch_departure(self)
  File "/home/melvin/.local/lib/python3.7/site-packages/sphinx/util/docutils.py", line 485, in dispatch_departure
    method(node)
  File "/home/melvin/.local/lib/python3.7/site-packages/sphinx/writers/latex.py", line 947, in depart_row
    underlined = [cell.row + cell.height == self.table.row + 1 for cell in cells]
  File "/home/melvin/.local/lib/python3.7/site-packages/sphinx/writers/latex.py", line 947, in <listcomp>
    underlined = [cell.row + cell.height == self.table.row + 1 for cell in cells]
AttributeError: 'NoneType' object has no attribute 'row'
> /home/melvin/.local/lib/python3.7/site-packages/sphinx/writers/latex.py(947)<listcomp>()
-> underlined = [cell.row + cell.height == self.table.row + 1 for cell in cells]
(Pdb)

Tables render the same as https://breathe.readthedocs.io/en/latest/tables.html with make html.

What Sphinx version do you have locally? I am thinking my ancient version is to blame here.

3.4.3

[utzig]

I tried with Sphinx==3.0.2 and it works fine as well, what docutils version do you have?

[vermeeren]

I tried with Sphinx==3.0.2 and it works fine as well, what docutils version do you have?

@utzig docutils 0.16

(Bit short on time, will have more proper reply and testing hopefully soon.)

[utzig]

I tried with Sphinx==3.0.2 and it works fine as well, what docutils version do you have?

@utzig docutils 0.16

(Bit short on time, will have more proper reply and testing hopefully soon.)

Same problem here! I found out the problem, Doxygen>=1.9.0 is required because the previous versions didn't have rowspan, colspan and align, so tables are not rendered correctly with those. Regarding the issues with "make all" I will see if I can fix it today or in the next few days.

[vermeeren]

Same problem here! I found out the problem, Doxygen>=1.9.0 is required because the previous versions didn't have rowspan, colspan and align, so tables are not rendered correctly with those.

@utzig Ah yes of course, my Doxygen is version 1.8.13 as I'm running Debian stable (buster).

Regarding the issues with "make all" I will see if I can fix it today or in the next few days.

I can't recall anything about this, is there some problem with this in master?

[utzig]

Same problem here! I found out the problem, Doxygen>=1.9.0 is required because the previous versions didn't have rowspan, colspan and align, so tables are not rendered correctly with those.

@utzig Ah yes of course, my Doxygen is version 1.8.13 as I'm running Debian stable (buster).

Regarding the issues with "make all" I will see if I can fix it today or in the next few days.

I can't recall anything about this, is there some problem with this in master?

I believe the issue is with my parser changes, or maybe rendering.

[utzig]

This is related to the same problem you suggested in the previous PR to use .. only: html (#638 (review)). I had previously removed that requirement on this PR but now I dropped the commit and it should finish "make all" again. If you have Doxygen>=1.9.0 (tbh I don't know which exact version, but 1.8.17 does not work!) it will even render the tables correctly, otherwise you get same results as before because there were no parameters in the XML to be started with. The latex writer I will have to investigate further to find out why it is breaking when there are spans in tables.

[utzig]

The failures in the Sphinx latex writer happen on this line: https://github.com/sphinx-doc/sphinx/blob/3.x/sphinx/writers/latex.py#L963

Since self.table.colcount is the number of the columns, when there is span at least some rows won't have colcount amount of columns and it will have None in the last element's in the list. So basically I think the latex writer does not support spanning, but I need to check with input directly from rST.

[vermeeren]

So basically I think the latex writer does not support spanning, but I need to check with input directly from rST.

@utzig Do you have some news about this? If not, I think we should merge this as-is keeping the .. only:: html to avoid broken PDF output with old Doxygen versions, as changes themselves are good improvements either way.

[utzig]

So basically I think the latex writer does not support spanning, but I need to check with input directly from rST.

@utzig Do you have some news about this? If not, I think we should merge this as-is keeping the .. only:: html to avoid broken PDF output with old Doxygen versions, as changes themselves are good improvements either way.

To correct myself, the Latex writer does actually support spans, the problem with the code I pointed previously is that it expects that valid spans exist, and in that case all correct data structures are built in memory; which is not possible if using invalid Doxygen XML (Doxygen<=1.8.18). I tried adding some fault handling for it last month, with a warning about the broken Doxygen XML, but wasn't very happy on how to handle it properly, so it's better to just render it wrong in this case and know about it. The html only build filters are already in place so this should be passing the CI and ready to merge.

[vermeeren]

@utzig Thanks for the details, I agree it's better to render wrong in these cases. Created #665 to raise the Doxygen dependency version after Debian bullseye is stable, from that point onwards anything older is considered legacy anyway.

Will merge and make a release in a sec.