Refactor distributions sagemath-{objects,categories} through sagemath-{environment,repl}

[mkoeppe]

Currently sagemath-categories distributes a superset of sagemath-objects, and both distributions include a copy of the doctester for their doctesting. We remove this duplication by declaring install-requires and extras-require.

The dependencies are illustrated in the updated developer's guide - https://e019add9a8540b8213dfc563e8e4bda04777282c--sagemath-tobias.netlify.app/developer/packaging_sage_library.html#hierarchy-of-distribution-packages

As #33817 is a dependency, this ticket can be tested by running

./bootstrap && make SAGE_CHECK=yes pypi-wheels

This test is also run as part of the "Build&Test" workflow.

CC: @kwankyu

Component: refactoring

Author: Matthias Koeppe

Branch/Commit: e5d1070

Reviewer: Kwankyu Lee

Issue created by migration from https://trac.sagemath.org/ticket/33812

[mkoeppe]

Changed dependencies from #29941, #28925 to #29941, #28925, #33817

[mkoeppe]

Branch: u/mkoeppe/refactor_distributions_sage_setup__sagemath__objects_categories__through_sagemath__environment_repl_

[sagetrac-git]

Commit: f6a2ee2

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

0b4d1fd	pkgs/sagemath-objects: Disable use of the Sage doctester
bbd9f05	build/pkgs/sagemath_objects/spkg-install: tox --force-dep does not work, remove
f6a2ee2	pkgs/sagemath-*/tox.ini: Add sagepython-norequirements-sagewheels-nopypi

[mkoeppe]

comment:5

Now that we are testing sagemath-objects with the stuff now in sagemath-environment stripped away, dependencies become apparent that need to be removed.

[sagemath_objects-9.6.rc3]   File "sage/misc/lazy_import.pyx", line 69, in init sage.misc.lazy_import (build/cythonized/sage/misc/lazy_import.c:10965)
[sagemath_objects-9.6.rc3]     from . import sageinspect
[sagemath_objects-9.6.rc3]   File "/Users/mkoeppe/s/sage/sage-rebasing/pkgs/sagemath-objects/.tox/sagepython-norequirements-sagewheels-nopypi/lib/python3.10/site-packages/sage/misc/sageinspect.py", line 122, in <module>
[sagemath_objects-9.6.rc3]     from sage.env import SAGE_LIB
[sagemath_objects-9.6.rc3] ModuleNotFoundError: No module named 'sage.env'

[mkoeppe]

Author: Matthias Koeppe

[mkoeppe]

comment:7

So this needs #33037 (Remove use of SAGE_LIB)

[mkoeppe]

Changed dependencies from #29941, #28925, #33817 to #29941, #28925, #33817, #33821

[mkoeppe]

Changed dependencies from #29941, #28925, #33817, #33821 to #29941, #28925, #33817, #33821, #33822

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

eef9e41	build/pkgs/python3/spkg-configure.m4: Check for module ensurepip
1340d95	build/pkgs/python3/distros/debian.txt: Add python3-venv
823bfd2	Merge #33822
ffed245	build/pkgs/sagemath_objects/spkg-install: Remove workaround for missing ensurepip

[sagetrac-git]

Changed commit from f6a2ee2 to ffed245

[sagetrac-git]

Changed commit from ffed245 to e3be906

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

e3be906

.github/workflows/build.yml: Install python3-venv

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

5f311f5

build/pkgs/sagemath_*/dependencies: Fix SAGE_CHECK dependencies

[sagetrac-git]

Changed commit from e3be906 to 5f311f5

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

c264e7f

build/make/Makefile.in: Handle SAGE_CHECK... for script packages

[sagetrac-git]

Changed commit from 5f311f5 to c264e7f

[mkoeppe]

Work Issues: Overriding SAGE_EDITABLE does not work

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

de18ba0

build/bin/sage-build-env[-config]: Do not override SAGE_EDITABLE if it is set already

[sagetrac-git]

Changed commit from c264e7f to de18ba0

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

202485f	Trac #33793: replace SAGE_TMP in SPYX_TMP.
827c527	Trac #33793: move spyx_tmp() to sage.misc.temporary_file.
fc25b79	Trac #33793: cache spyx_tmp() by hand.
3068676	Merge #33793
b41da0c	src/sage/misc/sageinspect.py: Remove use of SAGE_LIB
877fe16	src/sage/misc/cachefunc.pyx: Remove use of SAGE_LIB
8cacd5e	src/sage/misc/sageinspect.py: Remove import of SAGE_LIB
c8a5bdf	src/sage/misc/lazy_import.pyx: Import lazy_import_cache only when needed
cce5385	src/sage/sets/set_from_iterator.py: Remove use of SAGE_LIB
4f5f1f3	Merge #33821

[sagetrac-git]

Changed commit from de18ba0 to 4f5f1f3

[sagetrac-git]

Changed commit from 1fe31c8 to 9fd5817

[mkoeppe]

Changed dependencies from #33817 to none

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

e41e277

Merge tag '9.7' into t/33812/refactor_distributions_sage_setup__sagemath__objects_categories__through_sagemath__environment_repl_

[sagetrac-git]

Changed commit from 9fd5817 to e41e277

[kwankyu]

comment:60

Questions:

1. In dependencies files, sagemath-categories depends on cysignals gmpy2 sagemath_objects and sagemath-objects depends on cysignals gmpy2. Does cysignals gmpy2 need to be repeated?

2. In the hierarchy diagram, sagemath-repl depends on both sagemath-environment and sagemath-objects, but in the dependencies file of sagemath-repl spkg, I see sagemath_objects ipython. Why sagemath-environment is missing?

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

996461a	build/pkgs/sagemath_repl/dependencies: Update from pkgs/sagemath-repl/setup.cfg.m4
0ffdd57	pkgs/sagemath-repl/setup.cfg.m4: Update python-requires
141318f	build/pkgs/sagemath_categories/dependencies: Remove dependencies implied by sagemath_objects dependencies

[sagetrac-git]

Changed commit from e41e277 to 141318f

[kwankyu]

comment:62

Replying to Kwankyu Lee:

2. In the hierarchy diagram, sagemath-repl depends on both sagemath-environment and sagemath-objects, but in the dependencies file of sagemath-repl spkg, I see sagemath_objects ipython. Why sagemath-environment is missing?

Sorry.. The hierarchy diagram is the hierarchy of the distribution pkgs. I now see the correct dependencies in install_requires.

---

New commits:

996461a	build/pkgs/sagemath_repl/dependencies: Update from pkgs/sagemath-repl/setup.cfg.m4
0ffdd57	pkgs/sagemath-repl/setup.cfg.m4: Update python-requires
141318f	build/pkgs/sagemath_categories/dependencies: Remove dependencies implied by sagemath_objects dependencies

[mkoeppe]

comment:63

The distribution packages declare runtime dependencies (setup.cfg install_requires) and also build-time dependencies (pyproject.toml). I think in the diagram I only show the (covers of) runtime dependencies.

[kwankyu]

comment:64

Then perhaps the change of the commit 996461a is not strictly necessary? (if it is practically no-op and does not create any overhead, then no problem).

[mkoeppe]

comment:65

No, you spotted it correctly in comment:62. sagemath_environment was missing, and 996461a corrected that. (Also ipywidgets was missing.)

[kwankyu]

Changed branch from u/mkoeppe/refactor_distributions_sage_setup__sagemath__objects_categories__through_sagemath__environment_repl_ to u/klee/refactor_distributions_sage_setup__sagemath__objects_categories__through_sagemath__environment_repl_

[kwankyu]

Changed commit from 141318f to 3da1811

[kwankyu]

Changed reviewer from https://github.com/mkoeppe/sage/runs/7511007092 to Kwankyu Lee

[kwankyu]

New commits:

3da1811

Wrapping lines of README.rst

[kwankyu]

comment:68

What is the logic of the changes in lazy_import.pyx? I have no idea...

[kwankyu]

comment:69

Replying to Kwankyu Lee:

What is the logic of the changes in lazy_import.pyx? I have no idea...

It seems a provision to avoid ImportError in the situation when sage.misc.lazy_import is used when sage.features package is not present...

How about adding a comment to the code to explain the situation more concretely?

[mkoeppe]

Changed branch from u/klee/refactor_distributions_sage_setup__sagemath__objects_categories__through_sagemath__environment_repl_ to u/mkoeppe/refactor_distributions_sage_setup__sagemath__objects_categories__through_sagemath__environment_repl_

[kwankyu]

Changed commit from 3da1811 to e5d1070

[kwankyu]

comment:71

Thanks for the comment.

9.7b6 below is 9.7 in other files, though I think it doesn't matter as explained in the comment.

+++ b/build/pkgs/sagemath_repl/install-requires.txt
@@ -0,0 +1,2 @@
+# This file is updated on every release by the sage-update-version script
+sagemath-repl ~= 9.7b6

I tested this ticket in several ways. It works well. It looks good to me.

---

New commits:

e5d1070

src/sage/misc/lazy_import.pyx: Add comments

[mkoeppe]

comment:72

Thank you!

[vbraun]

Changed branch from u/mkoeppe/refactor_distributions_sage_setup__sagemath__objects_categories__through_sagemath__environment_repl_ to e5d1070