Package request: SageMath #19090

dkwo · 2020-02-13T09:20:50Z

https://www.sagemath.org
SageMath is a free open-source mathematics software system licensed under the GPL. It builds on top of many existing open-source packages: NumPy, SciPy, matplotlib, Sympy, Maxima, GAP, FLINT, R and many more. Access their combined power through a common, Python-based language or directly via interfaces or wrappers.
Mission: Creating a viable free open source alternative to Magma, Maple, Mathematica and Matlab.

leahneukirchen · 2020-02-13T19:20:39Z

Sage is a terrible mess to compile yourself. I recommend just using the Docker container.

dkwo · 2020-08-26T12:03:37Z

It is now possible to build Sage on Void.
More work is required to package the missing dependencies, and then install Sage (the library) as Python module.

motorto · 2020-09-02T09:08:18Z

Maybe do a check list of the missing packages to xbps-src , so that people could do them.
I can help !

dkwo · 2020-09-06T07:40:05Z

leahneukirchen · 2020-09-09T09:28:53Z

Sage is a great example of what Docker is useful for. *ducks and runs*

dkwo · 2021-02-24T09:59:56Z

I was previously able to build sage by installing these packages in Void:

CoinMP-devel
R
automake
boost-devel
cblas
cmake
ecl
ecm-devel
freetype-devel
gcc-fortran
gd-devel
gettext-devel
giac-devel
glpk-devel
gmpxx-devel
graphviz
harfbuzz
isl
libcurl-devel
libgda-devel
libgomp-devel
libmpc-devel
m4
mk-configure
mpc
mpir-devel
ninja
openblas-devel
pandoc
pari-devel
pari-elldata
pari-galdata
pari-galpol
pari-seadata
patch
pcre-devel
pcre2-devel
perl-Term-ReadLine-Gnu
pkgconf
ppl-devel
python3-Babel
python3-Pillow
python3-Sphinx
python3-devel
python3-matplotlib
python3-networkx
python3-pkgconfig
python3-scipy
python3-sympy
sqlite-devel
tox
yasm
zeromq-devel

	cblas
	harfbuzz
	libgda-devel
	mpir-devel
	python3-Babel
	python3-Pillow
	python3-Sphinx
	python3-devel
	python3-matplotlib
	python3-networkx
	python3-pkgconfig
	python3-scipy
	python3-sympy

tornaria · 2021-02-24T12:25:13Z

Note that sagemath includes (almost all) dependencies, and it's easy to compile with a minimal set of system packages:

gcc
make
m4
perl
binutils
git
tar
libgomp-devel

It's also recommended to install at least

gcc-fortran
libressl-devel

The difficult part would be to compile sagemath with as many system packages as possible.

dkwo · 2021-02-24T13:50:40Z

The difficult part would be to compile sagemath with as many system packages as possible.

Indeed. So that you don't get huge files and build times.

Eloitor · 2021-08-20T06:52:31Z

I would like to try to package sage, but I'm quite new to void. Where should I start?

dkwo · 2021-08-20T09:07:33Z

@Eloitor One could try to package some dependencies, see #19090 (comment)

tornaria · 2021-08-20T12:06:30Z

I would like to try to package sage, but I'm quite new to void. Where should I start?

It's a long road, we are slowly packaging dependencies but it will take some time.

It could be useful if you can test and review some pending PR: #29997, #30034, #30035, #30036.

If you merge those 4 PR, then you can compile sage with all of them installed (FlintQS, lcalc-devel, mpfi-devel, givaro-devel, fflas_ffpack, linbox-devel). The more dependencies we have, fewer have to be compiled by sage. Other recently merged dependencies that you may want to install are: gp2c, gf2x-devel, ntl-devel, flintlib-devel, arb-devel, eclib-devel. At this point I can have sage-9.4.rc2 compiled in ~ 25 min with -j8. More important, all those dependencies can now be used for other purposes.

I suggest that you focus on one or two packages and try to get them in good shape, pass QA, and get merged. It will take some time, but in my experience it's not a good idea to make too many half-baked PR at the same time. For example: try to get your lean-community package merged, and try to package m4ri and m4rie (both in one PR since they depend). Learn from that, e.g. how to test your templates in -musl and -i686, and also how to try cross compilation to see if it works, etc.

dkwo · 2021-08-23T16:47:09Z

@tornaria re:doctesting sage9.4, don't you get
sage -t --warn-long 28.3 --random-seed=0 src/sage/symbolic/integration/integral.py # 1 doctest failed
if you do make test? I got it a few days ago with rc2

tornaria · 2021-08-23T17:01:15Z

@tornaria re:doctesting sage9.4, don't you get
sage -t --warn-long 28.3 --random-seed=0 src/sage/symbolic/integration/integral.py # 1 doctest failed
if you do make test? I got it a few days ago with rc2

Yes, this is expected. I'm using the patch from https://trac.sagemath.org/ticket/32275 (commit dd75684).

dkwo · 2021-08-25T09:46:30Z

Just a note to self: python3-jupyter_ipywidgets in jupyterlab, for use with sage.

dkwo · 2021-08-29T11:20:24Z

@tornaria Do you also get this same list with sage 9.4, or is your list shorter :) ?

configure: notice: the following SPKGs did not find equivalent system packages:
brial cddlib cliquer fplll gfan libbraiding libhomfly lrcalc nauty palp planarity rw suitesparse symmetrica sympow tachyon zn_poly
_recommended coxeter3 graphviz igraph libnauty libsemigroups lrslib perl_cpan_polymake_prereq perl_mongodb

tornaria · 2021-08-29T13:03:53Z

@tornaria Do you also get this same list with sage 9.4, or is your list shorter :) ?

configure: notice: the following SPKGs did not find equivalent system packages:
brial cddlib cliquer fplll gfan libbraiding libhomfly lrcalc nauty palp planarity rw suitesparse symmetrica sympow tachyon zn_poly_recommended coxeter3 graphviz igraph libnauty libsemigroups lrslib perl_cpan_polymake_prereq perl_mongodb

fpll: I have a working template that I'll PR soon.
graphviz: you have to install graphviz (not graphviz-devel)
perl_cpan_polymake_prereq: need to install perl-XML-Writer, perl-XML-LibXML, perl-XML-LibXSLT, perl-File-Slurp, perl-JSON, perl-SVG (the last one is missing, I have a working template).

Are you able to compile sage on x86_64-musl ? I tried a few days ago and I the compilation stopped with an error in pyzmq.

dkwo · 2021-08-29T18:12:45Z

Thanks.
I have not tried on musl yet, as that would be my laptop, but I can try one of the coming days.

dkwo · 2021-09-02T13:59:41Z

@tornaria Same here on musl.

[pyzmq-22.0.3]   gcc build/temp.linux-x86_64-3.9/tmp/timer_createrk_w_nbj.o -o build/temp.linux-x86_64-3.9/a.out
[pyzmq-22.0.3]   error: [Errno 2] No such file or directory: 'a.out'
[pyzmq-22.0.3]   Building wheel for pyzmq (PEP 517): finished with status 'error'
[pyzmq-22.0.3]   ERROR: Failed building wheel for pyzmq
[pyzmq-22.0.3] Failed to build pyzmq
[pyzmq-22.0.3] ERROR: Failed to build one or more wheels
[pyzmq-22.0.3] Exception information:
[pyzmq-22.0.3] Traceback (most recent call last):
[pyzmq-22.0.3]   File "/tmp/build/sage-9.4/local/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 180, in _main
[pyzmq-22.0.3]     status = self.run(options, args)
[pyzmq-22.0.3]   File "/tmp/build/sage-9.4/local/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 204, in wrapper
[pyzmq-22.0.3]     return func(self, options, args)
[pyzmq-22.0.3]   File "/tmp/build/sage-9.4/local/lib/python3.9/site-packages/pip/_internal/commands/wheel.py", line 174, in run
[pyzmq-22.0.3]     raise CommandError(
[pyzmq-22.0.3] pip._internal.exceptions.CommandError: Failed to build one or more wheels
[pyzmq-22.0.3] Removed build tracker: '/tmp/pip-req-tracker-htt11etf'

I hope soon the python3-* packages and maxima will be usable from system, so that we don't have to deal with this issue.

tornaria · 2021-09-02T14:55:31Z

@tornaria Same here on musl.

Can you try again with the following patch?

diff --git a/build/pkgs/pyzmq/spkg-install.in b/build/pkgs/pyzmq/spkg-install.in
index 0ce404ee5a..b7260c6d27 100644
--- a/build/pkgs/pyzmq/spkg-install.in
+++ b/build/pkgs/pyzmq/spkg-install.in
@@ -1,6 +1,9 @@
 # Since we use environment vars we have to generate setup.cfg
 
-echo "[build_ext]" > src/setup.cfg
+echo "[global]" > src/setup.cfg
+echo "skip_check_zmq = True" >> src/setup.cfg
+
+echo "[build_ext]" >> src/setup.cfg
 
 # (I tried putting quotes around $SAGE_LOCAL to allow for spaces in
 # the path---which is never used but is a good habit to support---but

With this for me sage-9.4 compiles and tests ok on musl, except for the following doctest failures:

sage -t --random-seed=0 src/sage/geometry/polyhedron/base.py  # 1 doctest failed
sage -t --random-seed=0 src/sage/lfunctions/sympow.py  # 3 doctests failed
sage -t --random-seed=0 src/sage/symbolic/expression.pyx  # 1 doctest failed

tornaria · 2021-09-02T15:24:04Z

Just for the record: besides what it is already in PRs, I have working templates for

zn_poly
SuiteSparse
igraph
rankwidth
perl-SVG

What's left for compiling sage-9.4 with standard procedure is:

standard: brial cddlib cliquer gfan libbraiding libhomfly lrcalc nauty palp planarity symmetrica sympow tachyon
optional: coxeter3 libnauty libsemigroups lrslib perl_mongodb

Besides that, the remaining big packages seem to be gap and singular. They are both very useful independently of sage, so packaging those two is probably a worthwhile goal: sooner or later sage will use them from system, as well as maxima.

Moreover, when we get to package sage we probably won't be using the standard compile procedure but compile sage-as-a-python-library on top of all system-installed dependencies, I think that's what arch does and it's the way to move forward; we can't PR a monolith into void.

Eloitor · 2021-09-04T20:14:30Z

I've build Sage-9.4 using the pending PR. Now I would like to be able to call sage from python, for example using

from sage.all import *

but I get ModuleNotFoundError: No module named 'sage'
What can I do?

tornaria · 2021-09-04T20:47:54Z

I've build Sage-9.4 using the pending PR. Now I would like to be able to call sage from python, for example using
from sage.all import *
but I get ModuleNotFoundError: No module named 'sage'
What can I do?

That's not expected to work, because sage installs python packages (in particular sagelib) into its own python venv (should be in SAGE_LOCAL).

I think the simplest way for you is to run sage -python which will run your system python but with sage venv. Then your from sage.all import * should work.

Note that you won't have access to other python packages installed in your system. If you need python packages that are not installed in sage venv, you can use sage -pip to run pip in sage venv and install whatever you need there.

When we get around to make a sage package, we will install sagelib in the system python venv as any other python package in void (in fact, I think sage will be just a python package with lots of dependencies).

Eloitor · 2021-09-04T21:32:35Z

Ok, thanks for your answer.
I'm interested in installing sagelib in the system python venv because then I could use python debuggers... I'm not sure how to debug using sage -python.

tornaria · 2021-09-04T21:46:34Z

Can you install the python debugger in the sage venv? Try `sage -sh` for a shell with all environment setup for sage.

…

On Sat, Sep 4, 2021, 18:32 Eloi Torrents ***@***.***> wrote: Ok, thanks for your answer. I'm interested in installing sagelib in the system python venv because then I could use python debuggers... I'm not sure how to debug using sage -python. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#19090 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMCC27HHVUEXB2SYMDPZ5TUAKF73ANCNFSM4KUOAGWQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

Eloitor · 2021-09-05T10:05:02Z

Thanks I try to use the debugger in VSCode... In arch it works well with the default python debugger. It uses the command

/usr/bin/env /bin/python /home/eloi/.vscode/extensions/ms-python.python-2021.9.1191016588/pythonFiles/lib/python/debugpy/launcher 37247 -- "example_file.py"

So I tried adding debugpy to sage venv using

sage -sh
pip install debugpy

But I don't know how to continue. I guess I should replace /usr/bin/env with /path/to/sage-9.4/build/bin/sage-venv and /bin/python with path/to/sage-9.4/sage -python. But I'm not very experienced with python environments nor debugging with VSCode...

dkwo · 2021-09-06T09:59:17Z

Just for the record: besides what it is already in PRs, I have working templates for
* zn_poly

* SuiteSparse

* igraph

* rankwidth

* perl-SVG

If you make PRs for these, would you mind linking back here as well?

Moreover, when we get to package sage we probably won't be using the standard compile procedure but compile sage-as-a-python-library on top of all system-installed dependencies, I think that's what arch does and it's the way to move forward; we can't PR a monolith into void.

Totally agree. Btw, it seems sage itself is (slowly) proceeding towards this, by making sage-the library pip installable.

Ps. Will test your musl patch soon.

dkwo · 2021-09-07T11:28:43Z

Can you try again with the following patch?

For me the build on musl still fails

 ImportError: Error relocating /tmp/build/sage-9.4/local/lib/python3.9/site-packages/zmq/backend/cython/error.cpython-39-x86_64-linux-gnu.so: zmq_errno: symbol not found
  Building wheel for ipykernel (PEP 517): finished with status 'error'
  ERROR: Failed building wheel for ipykernel

but I'm not using the PR's on my laptop, only packaged ones, so that may be the cause.
(Either that, or newer musl I'm using.)

leahneukirchen · 2021-11-08T15:28:41Z

I merged a few PR, please tell how I can help else to make progress on this.

dkwo · 2021-11-08T17:17:27Z

@leahneukirchen Great! How about #32601 next?

tornaria · 2021-11-09T12:51:08Z

Unfortunately python3.10 is not yet supported, we should track https://trac.sagemath.org/ticket/30766

I'm testing the branch in that ticket (commit c73bc97), hopefully it will be merged soon.

leahneukirchen · 2021-11-09T15:32:41Z

We have all these bumps already, no?

dkwo · 2021-11-09T15:49:25Z

I guess yes (e.g. they're updating gmpy to work with python3.10, which we already have), so we only need those that affect sage the library (or stuff that we haven't packaged yet).

dkwo · 2021-11-09T15:52:57Z

@leahneukirchen How about #32641 next?

dkwo · 2021-11-10T09:05:47Z

@tornaria Do you think it's time to do gap? You mentioned you already have something, want to make a wip pr? The remaining packages all seem rather simple to package.

tornaria · 2021-11-10T19:58:23Z

@tornaria Do you think it's time to do gap? You mentioned you already have something, want to make a wip pr? The remaining packages all seem rather simple to package.

I'll search for my old gap package; I'm not using it, since it will not be used by sage and it seems quite tricky (there are tons of packages for gap). See https://trac.sagemath.org/ticket/29644 and https://trac.sagemath.org/ticket/31761.

It's a nice package to have, independently of sagemath.

At some point we should start figuring out what's the best way to package sagemath:
a. using the standard installation procedure, but using as many system packages as possible. This means sage runs its own local directory (and python venv)
b. installing it as a python module in the system. For this we need ALL dependencies to be installed and working (including all python dependencies), and presumably passing doctests will be harder because of differences in package versions or compile options. This is how it's done in arch (https://archlinux.org/packages/community/x86_64/sagemath/)

For (a) we don't need (can't use) a system gap. For (b) we require a system gap.

The remaining packages that sagemath will use from system if available are:

STANDARD: brial cliquer gfan libbraiding libhomfly lrcalc nauty palp tachyon
OPTIONAL: 4ti2 coxeter3 libnauty libsemigroups lrslib perl_mongodb
They are probably not very hard as in can be done in 15-20 minutes. Maybe tachyon and perl_mongodb are a bit harder.

I'll try to keep doing 1 or 2 a day from the standard list.

Here there's a template for a meta package "sage-deps" which will install all deps to build sage using all possible system packages.

https://github.com/tornaria/void-packages/blob/sage-deps/srcpkgs/sage-deps/template

leahneukirchen · 2021-11-10T20:27:06Z

I suggest we start with (a), because we need to do the work for that anyway, and making a template isn't hard, and we can ship something.

Ultimately, (b) would be nice to avoid duplication of all the python modules.

tornaria · 2021-11-10T20:33:39Z

I suggest we start with (a), because we need to do the work for that anyway, and making a template isn't hard, and we can ship something.

Agreed. Do you think we should install sage into /opt/sage/sage-${version}; this is similar to the old texlive pkg. Since sagemath is not relocatable/installable we might have to compile it in-place. At least we can try to have a working system sagemath, then we move forward from there to improve (and towards option b if possible).

leahneukirchen · 2021-11-10T20:40:51Z

I don't think /opt should be used for things we compile ourselves, how about /usr/lib/sage-${version}? (and symlink selected sage binaries to /usr/bin)

leahneukirchen · 2021-11-10T21:13:41Z

I think DESTDIR is supported? It doesn't really matter if there is no special do_install step...

dkwo · 2021-11-11T10:32:02Z

I agree, let's try with (a) first.

tornaria · 2021-11-11T14:13:17Z

See #34030

tornaria · 2021-11-22T03:53:42Z

Pending PRs: #34182, #34186, #34189, #34191, #34193.

dkwo · 2022-01-02T08:37:01Z

I'm closing this, as the relevant pr #34030 is almost good to go.

leahneukirchen mentioned this issue Jan 12, 2021

Please consider adding Sagemath void-linux/xbps#366

Closed

ericonr mentioned this issue Jan 13, 2021

Please consider adding Sagemath #27880

Closed

dkwo mentioned this issue Feb 24, 2021

giac does not support arbitrary precision floating point #28989

Closed

Eloitor mentioned this issue Aug 20, 2021

New math package: m4rie-20200125 #32601

Closed

3 tasks

dkwo closed this as completed Jan 2, 2022

prirai mentioned this issue Feb 18, 2023

Meta-ticket: spkg-configure: Try to use as many system packages as possible sagemath/sage#27330

Open

Package request: SageMath #19090

Package request: SageMath #19090

Comments

dkwo commented Feb 13, 2020 • edited Loading

leahneukirchen commented Feb 13, 2020

dkwo commented Aug 26, 2020 • edited Loading

motorto commented Sep 2, 2020

dkwo commented Sep 6, 2020 • edited Loading

leahneukirchen commented Sep 9, 2020

dkwo commented Feb 24, 2021 • edited Loading

tornaria commented Feb 24, 2021

dkwo commented Feb 24, 2021 • edited Loading

Eloitor commented Aug 20, 2021 • edited Loading

dkwo commented Aug 20, 2021

tornaria commented Aug 20, 2021

dkwo commented Aug 23, 2021

tornaria commented Aug 23, 2021

dkwo commented Aug 25, 2021

dkwo commented Aug 29, 2021 • edited Loading

tornaria commented Aug 29, 2021

dkwo commented Aug 29, 2021

dkwo commented Sep 2, 2021 • edited Loading

tornaria commented Sep 2, 2021

tornaria commented Sep 2, 2021

Eloitor commented Sep 4, 2021

tornaria commented Sep 4, 2021

Eloitor commented Sep 4, 2021

tornaria commented Sep 4, 2021 via email

Eloitor commented Sep 5, 2021 • edited Loading

dkwo commented Sep 6, 2021

dkwo commented Sep 7, 2021 • edited Loading

leahneukirchen commented Nov 8, 2021

dkwo commented Nov 8, 2021

tornaria commented Nov 9, 2021

leahneukirchen commented Nov 9, 2021 • edited Loading

dkwo commented Nov 9, 2021

dkwo commented Nov 9, 2021

dkwo commented Nov 10, 2021 • edited Loading

tornaria commented Nov 10, 2021

leahneukirchen commented Nov 10, 2021

tornaria commented Nov 10, 2021

leahneukirchen commented Nov 10, 2021

leahneukirchen commented Nov 10, 2021

dkwo commented Nov 11, 2021

tornaria commented Nov 11, 2021

tornaria commented Nov 22, 2021

dkwo commented Jan 2, 2022

dkwo commented Feb 13, 2020 •

edited

Loading

dkwo commented Aug 26, 2020 •

edited

Loading

dkwo commented Sep 6, 2020 •

edited

Loading

dkwo commented Feb 24, 2021 •

edited

Loading

dkwo commented Feb 24, 2021 •

edited

Loading

Eloitor commented Aug 20, 2021 •

edited

Loading

dkwo commented Aug 29, 2021 •

edited

Loading

dkwo commented Sep 2, 2021 •

edited

Loading

Eloitor commented Sep 5, 2021 •

edited

Loading

dkwo commented Sep 7, 2021 •

edited

Loading

leahneukirchen commented Nov 9, 2021 •

edited

Loading

dkwo commented Nov 10, 2021 •

edited

Loading