[docs] Fix Sphinx issue with linkless summaries

[johnkerl]

Issue and/or context: Ongoing work as part of #3282. Also fixes #3227.

Problem

What's right

• Go here: https://tiledbsoma.readthedocs.io/en/stable/python-tiledbsoma.html
• Scroll to "Classes"
• Pick one like tiledbsoma.Measurement
• Hover over that:

• This is a link; click it and you get here: https://tiledbsoma.readthedocs.io/en/stable/_autosummary/tiledbsoma.Measurement.html#tiledbsoma.Measurement

• Docstrings are present for the class -- full docstrings and not just the first summary sentence

What's wrong

• Simply go one level deeper: scroll down to "Methods"
• Hover over a method, e.g. add_new_collection
• This is not a hyperlink
• What you see is the first sentence of its docstring
• Nothing you can click on will show you the rest of the docstring

Analysis

At the first level we have hand-written .rst like this:

• https://tiledbsoma.readthedocs.io/en/stable/python-tiledbsoma.html
• https://github.com/single-cell-data/TileDB-SOMA/blob/1.15.0rc3/doc/source/python-tiledbsoma.rst

We keystroke out

.. autosummary::
    :toctree: _autosummary/

At the next level, we're letting autogen recurse for us and create .rst files. The .rst files that are auto-generated are missing this line:

    :toctree: _autosummary/

I spent the morning debugging, experimenting with sphinx-autogen and sphinx-apidoc, copy/pasting examples out of the Sphinx docs, playing with the :recursive: tag, doing pip install -U of every Sphinx package on my system, etc. -- and I came to the following conclusions:

• I could not find any syntax that would make the autogenerator emit the :toctree: dirnamegoeshere line on the .rst files it generates.
• Therefore I autogenerated the "next level in", manually edited those to include the :toctree: dirnamegoeshere lines, and am adding them to source control.

Changes:

As above.

Notes for Reviewer:

This means more manual editng when we create new classes/methods. However, there is already a non-zero amount of manual edits necessary: #3283 is a case in point. I would love to have fully automated/hands-off code -> doc generation but we simply do not have that at present. I will take a method with one more manual step, that produces correct and useful documentation, over a method one with one fewer manual step that produces incomplete and misleading documentation.

To view the docs: please compare

• https://tiledbsoma.readthedocs.io/en/stable/python-api.html# (old)
• https://tiledbsoma.readthedocs.io/en/kerl-fix-sphinx-linkless-summaries/python-api.html (new)

Note this second link is on what readthedocs calls a "hidden branch" which I created as a manual one-off for this PR review.

PR stacking as of today:

• main
• [docs] Add new things to .rst files #3283
• [docs] Fix Sphinx issue with linkless summaries #3284
• [python] Doc updates for new-shape feature #3285

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 85.40%. Comparing base (762abda) to head (00839da).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3284      +/-   ##
==========================================
+ Coverage   85.29%   85.40%   +0.10%     
==========================================
  Files          52       52              
  Lines        5517     5517              
==========================================
+ Hits         4706     4712       +6     
+ Misses        811      805       -6     

Flag	Coverage Δ
python	85.40% <ø> (+0.10%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
python_api	85.40% <ø> (+0.10%)	⬆️
libtiledbsoma	∅ <ø> (∅)

[ivirshup]

I will try to take a closer look later in the week and see if I can remember how to get autogenerated class docs working. But in the meantime, I noticed that the CSS for the class listing seems to be off in the demo build: https://tiledbsoma.readthedocs.io/en/kerl-fix-sphinx-linkless-summaries/python-tiledbsoma.html#classes

[johnkerl]

Thanks @ivirshup !

Can you please be more specific about

CSS for the class listing seems to be off in the demo build:

? I don't know what to try to fix ...

[ryan-williams]

Approving to unblock, will try to figure out how to get Sphinx to auto-generate the things we want it to, async…

[johnkerl]

Approving to unblock, will try to figure out how to get Sphinx to auto-generate the things we want it to, async…

This is great, thanks @ryan-williams !!

Also recall as I noted above, I don't entirely dislike having to spell more things out -- it lets us put methods in more intuitive order.

And honestly, the more I think about it, maybe I don't even really want fully autogenerated docs as much I had (very recently!) thought I did ... having an environment where manual curation is allowed and encouraged, I feel like this will improve the quality of the docs ......