Documentation rework

[zerothi]

We need to rework the documentation. This has been a long-standing wish, by everybody... :)

The purpose of this issue is to track documentation progress,
and to propose new documentation ideas.

It may take a long time to finalize, due to the limited amount of time we
have, but I hope that as many people can join in when we have a good picture of things, and contribute content.

This issue directly influences these issues:

• Documentation: signatures with typehints. #638
• Document categories. #674
• Documentation issues for SileBound #632
• type hint as much code as possible #455

Here is a list of things that can be tracked, individually

• The documentation needs to be more tutorial/guide focused
  • I kind of like the pandas site based on pydata-sphinx-theme which has nice blocks here and there (can be done using only sphinx-design)
  • Should we change theme? I know a lot of people really like RFD, but I have grown fond of the sphinx-book-theme (pandas uses pydata-sphinx-theme which is based on sphinx-book-theme.
  • We need to create some quick overview guides, here some suggestions:
    • Quick 10 minute introduction of what sisl can do for post-processing DFT data
    • Quick 10 minute introduction of what sisl can do for tight-binding calculations
    • In-depth Geometry class focus, how to manipulate, sort, use categories, etc.
    • In-depth SparseOrbital class focus, how to append, add, sub etc.
      It should be made clear here that all of these methods adheres to all Hamiltonian, DensityMatrix etc.
    • In-depth tutorial on how to access and see what can be read from Sile's.
    • Others? Please chip in so we can complete the list.
• Re-structure the API pages
  Have a look at other codes API structure, there is a lot of glue text to link the sections together, sisl misses that completely.
• Should we go and use the ipython directives, instead of nbsphinx?
  • ipython advantages includes, better search functionality, smaller, and faster documentation build times
    It also has the more familiar sphinx directives which can be used directly in the code.
  • nbsphinx advantages includes, direct download of the notebook (really nice), it can still be searched, but the notebooks has some problems creating links to the API documentation, which is really annoying. (I don't know if this can be fixed somehow)

The working plan would be:

1. Discuss the above details, the more feedback from devs/users the better.
2. Once we have agreed on what to do (the above list will be edited throughout the discussion) we can start doing stuff (no sooner!)
3. Wait for moving towards functional programming in sisl #665, Linear and efficient neighbour finder #393 and WIP: Added a matrix plot #668 to be merged.
4. Ping the community for help!
5. Start doing work 🚀

[zerothi]

@pfebrer I am particular eager to here about your take on the nbsphinx vs ipython directives. I am a bit annoyed that searching for the notebooks doesn't work seamlessly... but...

[pfebrer]

I find it much easier to write documentation in notebooks. Could you give an example of the searching problem? The only problem that I noticed for now is that names for functions/classes inside the notebook are not linked to the rest of the docs.

[zerothi]

I find it much easier to write documentation in notebooks. Could you give an example of the searching problem? The only problem that I noticed for now is that names for functions/classes inside the notebook are not linked to the rest of the docs.

Try and search for geometry, then you will find this 2nd output:

[GeometryPlot](https://zerothi.github.io/sisl/visualization/viz_module/showcase/GeometryPlot.html#GeometryPlot)

...not have a valid GeoJSON geometry.","Traces with locationmode *geojson-id* only support","*Polygon* and *MultiPolygon* geometries."].join(" "))}delete i[r]}switch(r.type){case"FeatureCollection":var h=r.features;for(n=0;n<h.length;n++)c(h[...

the context of the search is just not usable, and it is because of the representation of the full notebook...
I find this problematic since the search is indeed very useful.

And as you say the links to the rest of the document is very important (IMHO)!

[pfebrer]

Hmm that is plotly's javascript for the representation of the plot, not the notebook internals. Perhaps there are ways to avoid the search on that part.

Also now the whole plotly javascript library is included for each plot, if we could find a way to just include it once at the top (outside the notebook) it would make the pages much lighter. Also it might make it easier to make sphinx avoid that part when searching.

[zerothi]

which is annoying for the end user when searching for context ;)

[zerothi]

I think this requires some work to bypass.

[pfebrer]

which is annoying for the end user when searching for context ;)

Yes, but that would also be there if the plot is generated with an ipython directive. I'll try to fix it later.

[ialcon]

Hey guys,

I agree with this "I kind of like the pandas site based on pydata-sphinx-theme which has nice blocks here and there", as indeed, that kind of look appears as very structured and, consequently, easy to find what one is searching. I agree that part of the difficulty of the current documentation is finding what one needs at a particular time, and it is also true that the search engine does not work super well, because it shows many entries that do not seem to be uniquely the class that is being searched.

Finally, I also think that, at some point, having some sort of organization tree of sisl's classes/methods showing how everything is connected might also be useful. Part of the difficulty of the code (and documentation) is the (increasing) complexity of the thing, but having an overall view of how the different parts are connected in a panel-type view could help a lot the initial user to get a good qualitative picture of what sisl does. This of course, I guess should be done for the long term, as it seems to me it would require a significant amount of work.

My aim is to get in now seriously with sisl, but that also means I have a huge amount to learn... In the meantime I will try to help out as much as possible, but apologize if my contribution to all this (programming-wise) is, for now, limited. :-)

[zerothi]

Thanks Isaac! Input is most welcome, as always!

Ok, we can start playing around with some stuff, for starters we could play with the pydata-sphinx-theme. @pfebrer you had some comments on this framework wrt RTD?

[pfebrer]

It's just that I don't find it better. In fact I find it harder to read and navigate, being all white, without any visual guides for the brain.

But if everyone thinks it is better, what can I say 😅

[zerothi]

It has a dark-mode (which I kind of prefer)

[zerothi]

we can start with the basic pydata-sphinx-design principles, which shouldn't affect the theme, just the looks. Then we can always play afterwards.

[pfebrer]

It looks nice!

I would remove the "to be filled". Just leaving a blank space (without even the user guide title) may be better. SIESTA's documentation was also full of places that only said "TODO" and it looks very bad and is frustrating for the user.

[zerothi]

will be there in the next uploaded version

[zerothi]

And we are live!!! :)

[pfebrer]

I find it much easier to write documentation in notebooks. Could you give an example of the searching problem? The only problem that I noticed for now is that names for functions/classes inside the notebook are not linked to the rest of the docs.

Try and search for geometry, then you will find this 2nd output:

[GeometryPlot](https://zerothi.github.io/sisl/visualization/viz_module/showcase/GeometryPlot.html#GeometryPlot)

...not have a valid GeoJSON geometry.","Traces with locationmode *geojson-id* only support","*Polygon* and *MultiPolygon* geometries."].join(" "))}delete i[r]}switch(r.type){case"FeatureCollection":var h=r.features;for(n=0;n<h.length;n++)c(h[...

the context of the search is just not usable, and it is because of the representation of the full notebook... I find this problematic since the search is indeed very useful.

And as you say the links to the rest of the document is very important (IMHO)!

This is now fixed in sphinx! I created spatialaudio/nbsphinx#777, but it was a sphinx bug and it is now solved. So with the next sphinx release we should no longer see this text :)

[zerothi]

AMAZING!