Restructure docbuild as script packages sagemath_doc_html, sagemath_doc_pdf #31356

mkoeppe · 2021-02-07T18:33:03Z

The specific code in Makefile.in for the documentation and its dependencies is moved to the new script packages.

In preparation for #32759:

We introduce another installation tree (in addition to SAGE_LOCAL and SAGE_VENV), called SAGE_DOCS. When the installation tree variable is empty, we suppress building the package.
We introduce a mechanism to select an installation tree, the file trees.txt. Using this mechanism, we associate the new script packages with the installation tree SAGE_DOCS.

Next step:

./configure --disable-doc #32759 ./configure --disable-doc
pip-installable packages sagemath-doc-src, sagemath-doc-inventory, sagemath-doc-html, sagemath-doc-pdf #29868 pip-installable packages sagemath-doc-src, sagemath-doc-inventory, sagemath-doc-html, sagemath-doc-pdf
rename src/doc/bootstrap to build/pkgs/sagemath_doc_html/bootstrap.

Depends on #30383

CC: @jhpalmieri

Component: documentation

Author: Matthias Koeppe

Branch/Commit: fd298c4

Reviewer: John Palmieri

Issue created by migration from https://trac.sagemath.org/ticket/31356

The text was updated successfully, but these errors were encountered:

mkoeppe · 2021-02-07T19:43:45Z

Branch: u/mkoeppe/restructure_docbuild_as_optional_script_packages_sagemath_doc_html__sagemath_doc_pdf

mkoeppe · 2021-02-13T20:51:01Z

comment:2

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.

mkoeppe · 2021-02-25T04:56:44Z

Commit: 42b7714

mkoeppe · 2021-02-25T04:56:44Z

New commits:

`42b7714`	`WIP sagemath_doc_html sagemath_doc_pdf`

mkoeppe · 2021-02-25T04:56:44Z

Dependencies: #30010

sagetrac-git · 2021-02-25T05:40:53Z

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

`c262e44`	`build/pkgs/sage_docbuild/src: Update metadata`
`52778c3`	`build/pkgs/sage_docbuild/spkg-install: Use sdh_setup_bdist_wheel`
`b6abe5e`	`src/sage/docs/conf.py: Use absolute module names for sage_docbuild extensions`
`b42920b`	`trac 30010: (typo) change sage_doctest -> sage_docbuild`
`0a472f2`	`build/pkgs/sage_docbuild/src/setup.py: Restrict find_namespace_packages to sage_docbuild`
`1dfeb84`	`build/make/Makefile.in (sage_docbuild-clean): New, run it from build-clean`
`881c4af`	`src/sage_setup/__init__.py: Restore`
`4eca937`	`build/pkgs/sage_docbuild/src/setup.{cfg,py}: Use an explicit list of packages, add install_requires`
`aaab1d3`	`build/pkgs/sage_docbuild/dependencies: Add dependencies on .py files`
`f68feab`	`build/pkgs/sagemath_doc_{html,pdf}: New, delegate 'make doc-{html,pdf}' to here`

sagetrac-git · 2021-02-25T05:40:53Z

Changed commit from 42b7714 to f68feab

sagetrac-git · 2021-02-25T05:42:09Z

Changed commit from f68feab to 0e05512

sagetrac-git · 2021-02-25T05:42:09Z

Branch pushed to git repo; I updated commit sha1. New commits:

`0e05512`	`build/pkgs/sagemath_doc_html/dependencies: Add sage_docbuild`

sagetrac-git · 2021-02-25T05:48:22Z

Changed commit from 0e05512 to 5c19f46

sagetrac-git · 2021-02-25T05:48:22Z

Branch pushed to git repo; I updated commit sha1. New commits:

`d39fd70`	`build/pkgs/sagemath_doc_html: Add SPKG.rst, type`
`5c19f46`	`build/pkgs/sagemath_doc_html: Add package-version.txt`

mkoeppe · 2021-03-15T06:08:18Z

Changed dependencies from #30010 to #30383

sagetrac-git · 2021-03-15T06:10:10Z

Changed commit from 5c19f46 to 3cd70ff

sagetrac-git · 2021-03-15T06:10:10Z

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

`6dd12bd`	`build/pkgs/sagemath_doc_{html,pdf}: New, delegate 'make doc-{html,pdf}' to here`
`ec43b4c`	`build/pkgs/sagemath_doc_html/dependencies: Add sage_docbuild`
`862c6ac`	`build/pkgs/sagemath_doc_html: Add SPKG.rst, type`
`3cd70ff`	`build/pkgs/sagemath_doc_html: Add package-version.txt`

sagetrac-git · 2021-10-29T23:55:10Z

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

`d62fc48`	`WIP sagemath_doc_html sagemath_doc_pdf`

sagetrac-git · 2021-10-29T23:55:10Z

Changed commit from 3cd70ff to d62fc48

sagetrac-git · 2021-10-30T00:07:48Z

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

`0968c1c`	`WIP sagemath_doc_html sagemath_doc_pdf`

sagetrac-git · 2021-11-09T20:23:52Z

Branch pushed to git repo; I updated commit sha1. New commits:

`7989d87`	`src/doc/Makefile: Handle errors from './sage --docbuild --all-documents'`

sagetrac-git · 2021-11-09T20:23:52Z

Changed commit from eb65e03 to 7989d87

jhpalmieri · 2021-11-09T21:18:16Z

comment:34

I'm getting an error with the pdf build, one that I don't get with the develop branch: en/reference/plotting is failing, and the file local/share/doc/sage/latex/en/reference/plotting/plotting.log ends with

Package pdftex.def Info: plot-23.pdf  used on input line 864.
(pdftex.def)             Requested size: 462.52684pt x 346.89516pt.
<plot-24.pdf, id=486, 462.52798pt x 346.89601pt>
File: plot-24.pdf Graphic file (type pdf)
<use plot-24.pdf>
Package pdftex.def Info: plot-24.pdf  used on input line 884.
(pdftex.def)             Requested size: 462.52684pt x 346.89516pt.

[23 <./plot-22.pdf>] [24 <./plot-23.pdf>] [25 <./plot-24.pdf
!pdfTeX error: pdflatex (file ./plot-24.pdf): PDF inclusion: type <cmd> cannot 
be copied
 ==> Fatal error occurred, no output PDF file produced!

Any ideas?

mkoeppe · 2021-11-09T21:40:33Z

comment:35

No idea, I haven't seen this failure

sagetrac-git · 2021-11-09T21:52:59Z

Branch pushed to git repo; I updated commit sha1. New commits:

`d64fb76`	`build/make/Makefile.in: Undo reintroduction of SAGE_SKIP_PLOT_DIRECTIVE, SAGE_DOC_MATHJAX settings`
`6afc0d5`	`build/pkgs/sagemath_doc_html/spkg-install: Move handling of SAGE_DOC_JSMATH environment variable here from src/bin/sage-env`

sagetrac-git · 2021-11-09T21:52:59Z

Changed commit from 7989d87 to 6afc0d5

sagetrac-git · 2021-11-09T21:55:13Z

Changed commit from 6afc0d5 to b2cf40c

sagetrac-git · 2021-11-09T21:55:13Z

Branch pushed to git repo; I updated commit sha1. New commits:

`b2cf40c`	`build/make/Makefile.in: Undo reintroduction of SAGE_SKIP_PLOT_DIRECTIVE, SAGE_DOC_MATHJAX settings (fixup)`

jhpalmieri · 2021-11-09T23:03:10Z

comment:38

I'm also getting an error with the html build:

[developer] The HTML pages are in local/share/doc/sage/html/en/developer.
Error building the documentation.
Traceback (most recent call last):
  File "/usr/local/Cellar/[email protected]/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/Cellar/[email protected]/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.5.beta5/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage_docbuild/__main__.py", line 2, in <module>
    main()
  File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.5.beta5/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage_docbuild/__init__.py", line 1814, in main
    builder()
  File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.5.beta5/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage_docbuild/__init__.py", line 133, in f
    runsphinx()
  File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.5.beta5/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage_docbuild/sphinxbuild.py", line 323, in runsphinx
    sys.stderr.raise_errors()
  File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.5.beta5/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage_docbuild/sphinxbuild.py", line 258, in raise_errors
    raise OSError(self._error)
OSError: /Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.5.beta5/src/doc/en/developer/packaging.rst:721: WARNING: Duplicate explicit target name: "section-dependencies".

This is after make distclean && ./configure && make.

mkoeppe · 2021-11-10T00:05:17Z

comment:39

Thanks, I see this one now too, after make distclean

sagetrac-git · 2021-11-10T00:09:46Z

Branch pushed to git repo; I updated commit sha1. New commits:

`37a03bd`	`src/doc/en/developer/packaging.rst: Fix up section label`

sagetrac-git · 2021-11-10T00:09:46Z

Changed commit from b2cf40c to 37a03bd

sagetrac-git · 2021-11-10T00:21:26Z

Branch pushed to git repo; I updated commit sha1. New commits:

`fc89a66`	`build/pkgs/sagemath_doc_pdf/spkg-install: Set SAGE_DOC_MATHJAX as a workaround`

sagetrac-git · 2021-11-10T00:21:26Z

Changed commit from 37a03bd to fc89a66

mkoeppe · 2021-11-10T02:28:33Z

comment:42

PDF builds for me without error

jhpalmieri · 2021-11-10T05:28:43Z

comment:43

I think that sagemath_doc_html should be a dependency for sagemath_doc_pdf. Otherwise both may try to build at the same time, and they may both try to build the inventory files at the same time, which could easily lead to race conditions.

mkoeppe · 2021-11-10T05:34:26Z

comment:44

Yes, I agree

sagetrac-git · 2021-11-10T05:44:35Z

Changed commit from fc89a66 to fd298c4

sagetrac-git · 2021-11-10T05:44:35Z

Branch pushed to git repo; I updated commit sha1. New commits:

`fd298c4`	`build/pkgs/sagemath_doc_pdf/dependencies: Depend on sagemath_doc_html`

jhpalmieri · 2021-11-10T22:37:24Z

comment:46

Everything now builds for me.

mkoeppe · 2021-11-10T22:50:53Z

comment:47

Thanks for testing!

jhpalmieri · 2021-11-11T05:40:33Z

comment:48

I think this is ready to go. Merge now or wait until 9.6? I can imagine it breaking something for unforeseen reasons.

jhpalmieri · 2021-11-11T05:40:33Z

Reviewer: John Palmieri

mkoeppe · 2021-11-11T05:49:03Z

comment:49

I think we can merge it now

mkoeppe · 2021-11-11T17:24:13Z

comment:51

Thank you!

vbraun · 2021-11-14T17:06:30Z

Changed branch from u/mkoeppe/restructure_docbuild_as_optional_script_packages_sagemath_doc_html__sagemath_doc_pdf to fd298c4

mkoeppe added this to the sage-9.3 milestone Feb 7, 2021

mkoeppe added c: documentation labels Feb 7, 2021

mkoeppe modified the milestones: sage-9.3, sage-9.4 Feb 13, 2021

This comment has been minimized.

Sign in to view

mkoeppe changed the title ~~Restructure docbuild as optional script packages sagemath_doc_html, sagemath_doc_pdf~~ Restructure docbuild as script packages sagemath_doc_html, sagemath_doc_pdf Mar 15, 2021

This comment has been minimized.

Sign in to view

mkoeppe modified the milestones: sage-9.4, sage-9.5 Jul 19, 2021

jhpalmieri added s: positive review and removed s: needs review labels Nov 11, 2021

vbraun removed the s: positive review label Nov 14, 2021

vbraun closed this as completed in c962a2f Nov 14, 2021

Restructure docbuild as script packages sagemath_doc_html, sagemath_doc_pdf #31356

Restructure docbuild as script packages sagemath_doc_html, sagemath_doc_pdf #31356

Comments

mkoeppe commented Feb 7, 2021

mkoeppe commented Feb 7, 2021

mkoeppe commented Feb 13, 2021

mkoeppe commented Feb 25, 2021

mkoeppe commented Feb 25, 2021

mkoeppe commented Feb 25, 2021

sagetrac-git mannequin commented Feb 25, 2021

sagetrac-git mannequin commented Feb 25, 2021

sagetrac-git mannequin commented Feb 25, 2021

sagetrac-git mannequin commented Feb 25, 2021

sagetrac-git mannequin commented Feb 25, 2021

sagetrac-git mannequin commented Feb 25, 2021

This comment has been minimized.

mkoeppe commented Mar 15, 2021

This comment has been minimized.

sagetrac-git mannequin commented Mar 15, 2021

sagetrac-git mannequin commented Mar 15, 2021

This comment has been minimized.

This comment has been minimized.

sagetrac-git mannequin commented Oct 29, 2021

sagetrac-git mannequin commented Oct 29, 2021

sagetrac-git mannequin commented Oct 30, 2021

sagetrac-git mannequin commented Nov 9, 2021

sagetrac-git mannequin commented Nov 9, 2021

jhpalmieri commented Nov 9, 2021

mkoeppe commented Nov 9, 2021

sagetrac-git mannequin commented Nov 9, 2021

sagetrac-git mannequin commented Nov 9, 2021

sagetrac-git mannequin commented Nov 9, 2021

sagetrac-git mannequin commented Nov 9, 2021

jhpalmieri commented Nov 9, 2021

mkoeppe commented Nov 10, 2021

sagetrac-git mannequin commented Nov 10, 2021

sagetrac-git mannequin commented Nov 10, 2021

sagetrac-git mannequin commented Nov 10, 2021

sagetrac-git mannequin commented Nov 10, 2021

mkoeppe commented Nov 10, 2021

jhpalmieri commented Nov 10, 2021

mkoeppe commented Nov 10, 2021

sagetrac-git mannequin commented Nov 10, 2021

sagetrac-git mannequin commented Nov 10, 2021

jhpalmieri commented Nov 10, 2021

mkoeppe commented Nov 10, 2021

jhpalmieri commented Nov 11, 2021

jhpalmieri commented Nov 11, 2021

mkoeppe commented Nov 11, 2021

mkoeppe commented Nov 11, 2021

vbraun commented Nov 14, 2021