Improve gap_packages

[vbraun]

We should include more packages into GAP. This ticket adds the following:

• cohomolo
• CoReLG
• GBNP
• hecke
• liealgdb
• LiePRing
• LieRing
• loops
• mapclass
• qpa
• quagroup
• repsn
• sla

new tarball

CC: @dimpase @miguelmarco @vbraun

Component: packages: optional

Keywords: days85

Author: Travis Scrimshaw

Branch/Commit: 951e7e7

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/13426

[tscrim]

comment:6

See also #22190 and #22244.

[tscrim]

Branch: public/packages/increase_gap_packages-22244

[tscrim]

New commits:

951e7e7

Added new packages to gap_packages.

[tscrim]

Author: Travis Scrimshaw

[tscrim]

Commit: 951e7e7

[tscrim]

Changed keywords from none to days85

[dimpase]

comment:9

to quote email by Alex K. on a GAP developers list:

In pull request https://github.com/gap-system/gap/pull/1135
I have attempted to reduce a list of packages loaded by default to

[ "autpgrp", "ctbllib", "factint", "fga", "irredsol", "tomlib" ],

With dependencies, this results in the following set of
packages being loaded at startup:

AtlasRep 1.5.1, AutPGrp 1.6, Browse 1.8.6, CRISP 1.4.4, CTblLib 1.2.2,
FactInt 1.5.3, FGA 1.3.1, GAPDoc 1.5.1, IO 4.4.6, IRREDSOL 1.3.1,
SpinSym 1.5, TomLib 1.2.6

so we should aim to have as many from the latter list as possible.
(packages that need GAP kernel extensions, like IO, are not possible to support ATM, as this will need changes in libGAP).

[kiwifb]

comment:10

AtlasRep is problematic in global installs. Debian has a package for it that does something but I may miss some bits that they put in gap itself. The problem is that AtlasRep fetch data from the net and tries to dump it where the package is installed. If you are a regular user and gap is installed in location belonging to root you are in trouble.

AtlasRep really need to dump data in ~/.gap.

[tscrim]

comment:11

To note, we already have (some form of) AtlasRep, AutPGrp, CTblLib, FactInt included in gap_packages.

The changes to libGAP that will allow IO should be handled in #22626 (or at least be doable afterwards). However, I don't want to hold up the addition of some other packages, a number of which are related to my research (I'm already using QuaGroup in #22623). So I'd appreciate it if we could push off making gap_packages "perfect" to a later ticket.

[tscrim]

comment:12

Let me reformulate that better as a question: What would you want to see included that currently is not?

[tscrim]

comment:16

The bigger thing for me right now is QuaGroup. I got an e-mail where I could point towards #22623, but had some issues with telling them how to get the QuaGroup in a robust way.

I agree completely with points 1 and 2. For point 3, I feel that gap_packages is the closest thing that Sage has to an "official" way to install optional GAP pacakges because I think Sage tells users " you do not have to know anything about the underlying pacakges". So this would be a change to the official way, but I think in a good way.

Some other technical questions:

• What if we want a GAP package to only be for one version of Sage, what should the official way to install it (do we even care)?

• On my system, I do not have a ~/.gap folder (of course, I can create one), but I do not have a non-Sage copy of GAP. Do we want this in ~/.gap or some subfolder of ~/.sage (in particular, ~/.sage/gap)? At least, I don't think your proposal will work because:

  sage: gap.console()
  gap> Print( GAPInfo.RootPaths, "\n"); 
  [ "/home/travis/sage-build/local/gap/latest/" ]
  

• How do we get users to run gap_reset_workspace() (IIRC it is necessary)?

[kiwifb]

comment:17

Replying to @tscrim:

The bigger thing for me right now is QuaGroup. I got an e-mail where I could point towards #22623, but had some issues with telling them how to get the QuaGroup in a robust way.

I agree completely with points 1 and 2. For point 3, I feel that gap_packages is the closest thing that Sage has to an "official" way to install optional GAP pacakges because I think Sage tells users " you do not have to know anything about the underlying pacakges". So this would be a change to the official way, but I think in a good way.

Some other technical questions:

• What if we want a GAP package to only be for one version of Sage, what should the official way to install it (do we even care)?

I am not sure we can enforce that. At the very least the version of gap associated with that version of sage would need to be unique.

• On my system, I do not have a ~/.gap folder (of course, I can create one), but I do not have a non-Sage copy of GAP. Do we want this in ~/.gap or some subfolder of ~/.sage (in particular, ~/.sage/gap)? At least, I don't think your proposal will work because:

  sage: gap.console()
  gap> Print( GAPInfo.RootPaths, "\n"); 
  [ "/home/travis/sage-build/local/gap/latest/" ]
  

After inspection, it seems to be related to how gap is started from sage. That will need to be inspected.

fbissey@moonloop ~/sandbox/git-fork/sage $ ./sage -sh

Starting subshell with Sage environment variables set.  Don't forget
to exit when you are done.  Beware:
 * Do not do anything with other copies of Sage on your system.
 * Do not use this for installing Sage packages using "sage -i" or for
   running "make" at Sage's root directory.  These should be done
   outside the Sage shell.

Bypassing shell configuration files...

Note: SAGE_ROOT=/home/fbissey/sandbox/git-fork/sage
(sage-sh) fbissey@moonloop:sage$ gap
 ┌───────┐   GAP 4.8.6, 12-Nov-2016, build of 2017-07-06 10:42:48 (NZST)
 │  GAP  │   http://www.gap-system.org
 └───────┘   Architecture: x86_64-pc-linux-gnu-gcc-default64
 Libs used:  gmp, readline
 Loading the library and packages ...
 Packages:   GAPDoc 1.5.1
 Try '??help' for help. See also '?copyright', '?cite' and '?authors'
gap> Print( GAPInfo.RootPaths, "\n");
[ "/home/fbissey/.gap/", "/home/fbissey/sandbox/git-fork/sage/local/gap/latest/" ]

Related to your previous question, RootPaths can be customised (and obviously it is done in sage already), so you could imagine adding a unique location for that version of sage.

• How do we get users to run gap_reset_workspace() (IIRC it is necessary)?

I am aware of that problem and I am not sure. I wonder if upstream sees this as a problem or if it is particular to the way sage interface with gap.

[tscrim]

comment:18

Replying to @kiwifb:

Replying to @tscrim:

• What if we want a GAP package to only be for one version of Sage, what should the official way to install it (do we even care)?

I am not sure we can enforce that. At the very least the version of gap associated with that version of sage would need to be unique.

I guess we already have this sort of problem with multiple Sage versions and init.sage being in ~/.sage. So putting something in ~/.sage would not exacerbate the problem.

• On my system, I do not have a ~/.gap folder (of course, I can create one), but I do not have a non-Sage copy of GAP. Do we want this in ~/.gap or some subfolder of ~/.sage (in particular, ~/.sage/gap)? At least, I don't think your proposal will work because:

  sage: gap.console()
  gap> Print( GAPInfo.RootPaths, "\n"); 
  [ "/home/travis/sage-build/local/gap/latest/" ]
  

After inspection, it seems to be related to how gap is started from sage. That will need to be inspected.

fbissey@moonloop ~/sandbox/git-fork/sage $ ./sage -sh
[snip]
gap> Print( GAPInfo.RootPaths, "\n");
[ "/home/fbissey/.gap/", "/home/fbissey/sandbox/git-fork/sage/local/gap/latest/" ]

Doing this, I get:

gap> Print( GAPInfo.RootPaths, "\n");
[ "/home/travis/.gap/", "/home/travis/sage-build/local/gap/latest/" ]

However, via gap.console() might be considered a special way of starting gap, but it is something I think we will need to address because

sage: %gap

  --> Switching to Gap <--

gap: Print( GAPInfo.RootPaths, "\n");
[ "/home/travis/sage-build/local/gap/latest/" ]

sage: gap('GAPInfo.RootPaths')
[ "/home/travis/sage-build/local/gap/latest/" ]

So within Sage calls to gap, it seems to use that version.

Related to your previous question, RootPaths can be customised (and obviously it is done in sage already), so you could imagine adding a unique location for that version of sage.

I propose ~/.sage/gap.

I'm guessing anything installed as proposed would also be available via libgap. At least via gap_packages in #22623 I was able to make calls via libgap. (Granted, I think this will be made moot when we go to the next gap version).

• How do we get users to run gap_reset_workspace() (IIRC it is necessary)?

I am aware of that problem and I am not sure. I wonder if upstream sees this as a problem or if it is particular to the way sage interface with gap.

I have no idea. At least if we had some way to install this using something like sage -i, we could run it as part of our installation process.

[kiwifb]

comment:19

Replying to @tscrim:

Replying to @kiwifb:

Replying to @tscrim:

• What if we want a GAP package to only be for one version of Sage, what should the official way to install it (do we even care)?

I am not sure we can enforce that. At the very least the version of gap associated with that version of sage would need to be unique.

I guess we already have this sort of problem with multiple Sage versions and init.sage being in ~/.sage. So putting something in ~/.sage would not exacerbate the problem.

I may have misunderstood you. You meant a gap package only available to sage, while with your wording I understood sage-X.Y. Is it correct?

So far I haven't been able to dig where sage fiddle with the gap RootPaths.

[tscrim]

comment:20

Replying to @kiwifb:

Replying to @tscrim:

Replying to @kiwifb:

Replying to @tscrim:

• What if we want a GAP package to only be for one version of Sage, what should the official way to install it (do we even care)?

I am not sure we can enforce that. At the very least the version of gap associated with that version of sage would need to be unique.

I guess we already have this sort of problem with multiple Sage versions and init.sage being in ~/.sage. So putting something in ~/.sage would not exacerbate the problem.

I may have misunderstood you. You meant a gap package only available to sage, while with your wording I understood sage-X.Y. Is it correct?

Let me try to be more precise. Suppose I have two versions of sage, X and Y. I want to be able to install gap packages only available to X but not to Y.

Actually, thinking about this more, I don't think this would be a problem as if a gap package doesn't work with a particular version of gap, then it will fail normally. Plus there are probably better ways to compare two versions of a package. So I don't think this is really necessary.

[dimpase]

comment:21

A clean way to resolve this seems to be to pass the token upstream: ask them for a variant of gap -i facility to install GAPs packages.

I have opened up an issue #1499 on GAP's github repo.

[jdemeyer]

comment:22

Replying to @tscrim:

At least if we had some way to install this using something like sage -i, we could run it as part of our installation process.

-1 to use sage -i to install packages in ~/.sage/gap or some variant of that. I believe that there should be a clear difference between installing packages inside Sage itself and inside some user directory.

[kiwifb]

comment:23

OK after some rather intricate tracking I found why RootPaths is different in sage from outside sage: https://github.com/sagemath/sage-prod/blob/master/src/sage/interfaces/gap.py#L200

The "-r" parameter passed to gap has the effect to make it ignore any .gaprc file and set IgnoreGapRC. But a side effect is that if IgnoreGapRC is set

src/system.c:        if (!IgnoreGapRC) {
src/system.c-          SySetGapRootPath(DotGapPath);
src/system.c-        }

.gap is not added to RootPaths.

[tscrim]

comment:24

What we would be doing is setting a custom place for the Sage version of gap packages, and and so we would need a custom root path passed in via gap -l <path>.

@jdemeyer I somewhat see this as installing a Sage package, just within the gap part of Sage. However, I am not really convinced of the feasibility of sage -i as we would need all of the metadata for the gap packages and mirrors and other stuff that (IMO) is not really worth dealing with. I can deal with installing it in a specific folder (or making that the official installation method). Yet I see ~/.sage as being something (mildly) hidden, and so we shouldn't need to tell users to go into there. IDK. What would be your preference?

Upstream does seem amenable to having gap -i, but not enough people with the time and/or skill to properly implement it. Also, we would want to have gap install it to a specified place, which would not be where gap is installed for Sage as they would get removed on upgrade. I would like to have something more workable in the near future.

[jdemeyer]

comment:25

Replying to @tscrim:

What would be your preference?

What's wrong with keeping the current $SAGE_LOCAL/gap/latest? We install every Sage package in $SAGE_LOCAL, why would GAP packages be an exception?

[dimpase]

comment:26

Replying to @jdemeyer:

Replying to @tscrim:

What would be your preference?

What's wrong with keeping the current $SAGE_LOCAL/gap/latest? We install every Sage package in $SAGE_LOCAL, why would GAP packages be an exception?

Note that pip --user would install Python modules somewhere else.

[dimpase]

comment:27

Replying to @tscrim:

Upstream does seem amenable to having gap -i, but not enough people with the time and/or skill to properly implement it. Also, we would want to have gap install it to a specified place, which would not be where gap is installed for Sage as they would get removed on upgrade. I would like to have something more workable in the near future.

I'm going to work on gap -i; it appears to align well with the current ODK grant goals.

[tscrim]

comment:28

Replying to @jdemeyer:

Replying to @tscrim:

What would be your preference?

What's wrong with keeping the current $SAGE_LOCAL/gap/latest? We install every Sage package in $SAGE_LOCAL, why would GAP packages be an exception?

The problem as I understand it is that $SAGE_LOCAL/gap/latest is just a symlink and changes on each upgrade. So suppose you install a package P in gap-X, then you upgrade (in the process of upgrading Sage) to gap-Y. Then $SAGE_LOCAL/gap/latest points to gap-Y, which does not contain P, and your code that uses the package suddenly doesn't work. With the current gap_packages, this is not a problem because our current framework automatically updates that as well. Or am I misunderstanding something?

However, I was asking about your preference for how to tell users to install optional gap packages? I guess your proposal it to tell them untar in $SAGE_LOCAL/gap/latest and make that the official way (rather than on a mostly hidden sage wiki page).

[tscrim]

comment:29

Replying to @dimpase:

Replying to @tscrim:

Upstream does seem amenable to having gap -i, but not enough people with the time and/or skill to properly implement it. Also, we would want to have gap install it to a specified place, which would not be where gap is installed for Sage as they would get removed on upgrade. I would like to have something more workable in the near future.

I'm going to work on gap -i; it appears to align well with the current ODK grant goals.

Great. I would offer to help, but I have neither the time nor expertise to contribute much. However, let me know if there is anything I can do. I guess if that is in the works, then we can simply tell users to install in a specific place until that method is realized as a workaround.

[mantepse]

comment:30

What is the status of this ticket? It appears to work, and I'd like to have qpa easily accessible. Can I have some cake?

[dimpase]

comment:31

So these added packages don't do any GAP kernel modules, right?
Just checking---otherwise it looks good.

[dimpase]

Reviewer: Dima Pasechnik

[tscrim]

comment:32

I believe so. The only thing that needs to be compiled is cohomolo (https://www.gap-system.org/Packages/cohomolo.html), but that is because of some extra C bindings IIRC.

[dimpase]

comment:33

OK, send it to the bots then. I hope they can get the tar file.

[tscrim]

comment:34

Volker, is there a reason why this ticket did not get merged in the past beta?

[tscrim]

Changed dependencies from #13211 to none

[vbraun]

Changed branch from public/packages/increase_gap_packages-22244 to 951e7e7