Extended primitive groups library for degrees up to 4095.

[ChristopherRussell]

Please make sure that this pull request:

• is submitted to the correct branch (the stable branch is only for bugfixes)
• contains an accurate description of changes for the release notes below
• provides new tests or relies on existing ones
• correctly refers to other issues and related pull requests

Description of changes (for the release notes)

The groups were taken from the MAGMA database created by Coutts, Roney-Dougal and Quick in 2009. The conversion was performed by using MAGMA to output GAP readable database files. Two tests were performed to ensure accuracy by comparing the created database to the MAGMA database. First, for each degree, the number of groups of each O'Nan-Scott type was compared with MAGMA. Second, for each degree the list of orders of groups at that degree were compared with those in MAGMA.

To allow the new database files to be accessed three parameters in primitiv.grp were also changed:
PRIMRANGE - List of degrees for which the database has entries
PRIMINDEX - List of integers where, for example, entry i being 27 tells gap that the information for primitive groups of degree i is stored in gps27.g
PRIMLENGTHS - List of integers where the i'th entry is the number of primitive groups of degree i in the database.

This work was carried out as part of my Senior Honours Project (MMath) at St Andrews. The project was titled 'Constructing a Database of Primitive Permutation Groups' and I was supervised by Colva Roney-Dougal.

[markuspf]

• Is this really a change for stable-4.8, and not for master (i.e., is this a bugfix?)
• Did anyone try running the primitive groups database test with the bigger database?

[markuspf]

Also, thanks @ChristopherRussell great work!

[olexandr-konovalov]

Thank @ChristopherRussell ! Also, for a very good description!

@markuspf: as we discussed during the call today, let' have it submitted to stable-4.8 branch. This extension is equivalent to updating a package, only the "package" is still a part of the core system. In case of any difficulties, we can always easily merge this into master instead of stable. But if it fits well into the system, there would be no obstacles for including it in one of the next minor releases. Also, this is better in a view of planned separation of the primitive groups library into a separate package, since this avoids the need of synchronising the work that @ChristopherRussell had already done with the moving target.

I've put a collection of some TODOs to review and merge this PR:

• increment version number for the prim component
• update descriptions in Chapter "Group Libraries", section "Primitive Permutation Groups" (and eventually on the GAP website)
• run extended version of the primsan.tst (the current version checks only degrees in [2..999]).

Check the behaviour of the following:

• PrimitiveGroup
• OnePrimitiveGroup
• NrPrimitiveGroups
• AllPrimitiveGroups
• PrimitiveGroupsIterator
• COHORTS_PRIMITIVE_GROUPS
• PrimitiveIdentification
• SimsNo
• PRIMITIVE_INDICES_MAGMA

[markuspf]

The test primsan.tst passes on my machine in roughly 1100 seconds for what it's worth.

[markuspf]

I don't agree with bigger updates within stable-4.8 btw, but I am probably too late in the game for this.

[markuspf]

Also, apparently I am stupid, because primsan.tst only tests up to degree 999...

[ChristopherRussell]

Also thanks @alex-konovalov for the help submitting this pull request!

Re checking the behaviour of the following:
COHORTS_PRIMITIVE_GROUPS - I did some work on this before realising it wasn't part of my project. If you want me to then I could finish it.
PRIMITIVE_INDICES_MAGMA - I haven't updated this function but I know the order that the new primitive groups are stored in GAP matches the order that they are stored in MAGMA.

[olexandr-konovalov]

@ChristopherRussell : I've figured out that PRIMITIVE_INDICES_MAGMA is a global variable - it is a list of length 999 because, as GAP manual says, "For historical reasons, its indexing up to degree 999 differs from the one used by GAP". Thus, if the order in which the groups are stored in GAP and MAGMA is the same, there is nothing to do here.

As for COHORTS_PRIMITIVE_GROUPS, this is indeed a list of length 2499, so it should be either documented that it does not cover larger degrees, or it could be extended. It should be possible to autogenerate the code from GAP, so if you would be willing to do that, I will be great. We can leave it to run on the server for a while.

I've checked that COHORTS_PRIMITIVE_GROUPS is not used anywhere in GAP or currently redistributed packages, but that's likely useful for some other reasons.

[olexandr-konovalov]

I've assigned this to myself and started to test. NrPrimitiveGroups checked:

gap> Sum(List([1..4095],NrPrimitiveGroups));
24558

Now running extended version of primsan.tst for all degrees up to 4095. Here is what it does:

checkdegree := function(n) 
local g;
for g in AllPrimitiveGroups(DegreeOperation,n) do
  if MovedPoints(g) <> [1..n] or not IsTransitive(g,[1..n]) or not IsPrimitive(g,[1..n]) then
    Error("Failure at ",g," degree ",n,"\n");
  fi;
od;
end;;
for n in [2..4095] do 
  Print("Testing degree ", n, "\n");
  checkdegree(n);
od;

[olexandr-konovalov]

Testing with extended primsan.tst is underway, this is where it is after 16 hours:

Testing degree 2684
#I  computing Conway polynomial for p = 2683 and n = 1
Testing degree 2685
Testing degree 2686
Testing degree 2687
Testing degree 2688
#I  computing Conway polynomial for p = 2687 and n = 1

P.S. After 25 hours:

Testing degree 2998
Testing degree 2999
Testing degree 3000
#I  computing Conway polynomial for p = 2999 and n = 1
Testing degree 3001
Testing degree 3002
#I  computing Conway polynomial for p = 3001 and n = 1
Testing degree 3003

P.P.S. Passed groups of degree 3500 after about 48 hours, and was testing degree 3866 after 72 hours.

[olexandr-konovalov]

Extended version of primsan.tst completed successfully after 90 hours! Next to testing some remaining functions. I've just started the test which exercises NrPrimitiveGroups, PrimitiveGroup and PrimitiveIdentification:

checkprimid := function(n) 
local i,g,h;
for i in [1..NrPrimitiveGroups(n)] do
  Print(i,"/",NrPrimitiveGroups(n),"\r");
  g:=PrimitiveGroup(n,i);
  h:=Group(GeneratorsOfGroup(g));
  if PrimitiveIdentification(h) <> i then
    Error("Failure at PrimitiveGroup(",n,",",i,"\n");
  fi;
od;
Print("\n");
end;;

for n in [2..4095] do 
  Print("Testing degree ", n, "\n");
  checkprimid(n);
od;

[olexandr-konovalov]

After 6 hours, GAP started with -o 8g run out of memory at degree 1728. Restarting with larger setting.

[olexandr-konovalov]

Tested OnePrimitiveGroup with

List([2..4095],i->OnePrimitiveGroup(NrMovedPoints,i));

• everything works fine. The test took about 100 seconds (requires computing a number of Conway polynomials).

[olexandr-konovalov]

This test exercises PrimitiveGroup and PrimitiveGroupsIterator. It took 565 seconds:

checkprimiter := function(n) 
local i,g,h,iter;
iter:=PrimitiveGroupsIterator(DegreeOperation,n);
i:=0;
for g in iter do
  i:=i+1;
  h:=PrimitiveGroup(n,i);
  if g<>h then
    Error("Failure at PrimitiveGroup(",n,",",i,"\n");
  fi;
od;
end;;

for n in [2..4095] do 
  Print("Testing degree ", n, "\n");
  checkprimiter(n);
od;

[olexandr-konovalov]

I have used MAGMA online calculator to get the number of primitive groups of all degrees up to 4095:

[NumberOfPrimitiveGroups(i) : i in [1..4095]]

All number agreed with those already in the primitive group library or provided by this pull request, except that GAP reports that there are zero primitive groups of degree one, and MAGMA says there is one.

[olexandr-konovalov]

I think this PR is ready to be merged, as the testing did not detect any problems with things that ARE in this PR. After that, the plan would be to:

• increment version number for the prim component
• update descriptions in Chapter "Group Libraries", section "Primitive Permutation Groups" (and eventually on the GAP website)
• resolve Range of restricted degrees in AllPrimitiveGroups #832, Recursion depth trap in SimsNo  #833 and Problems with PrimitiveIdenfification #834.
• move Primitive Groups Library to PrimGrp package at https://github.com/gap-packages/primgrp
• COHORTS_PRIMITIVE_GROUPS may be added at some later point, documentation should mention that they are available only for degree<2500.

I am now starting to agree with @markuspf that this PR should be merged into master, not into stable-4.8

[olexandr-konovalov]

P.S. the version prim 2.1 -> prim 3.0 could be actually incremented while merging the PR. The subsequent package may have version 3.1.

[markuspf]

As discussed with @alex-konovalov I just cherry-picked this onto master. This makes @ChristopherRussell the top GAP contributor on paper on github with almost 300k added lines ;).

[markuspf]

Closed as cherry-picked onto master.

[olexandr-konovalov]

@markuspf @ChristopherRussell thanks - now I will increment version number for the prim component and update descriptions in Chapter "Group Libraries", section "Primitive Permutation Groups" (and eventually on the GAP website), and also look at #832, #833 and maybe #834.

[ChristopherRussell]

There is an issue with degree 3600 (contained in prim/grps/gps35.g). I believe the only problem is that many of the groups have the wrong O'Nan-Scott type. I am away for 6 weeks without my laptop so I won't be able to fix this until the end of August.

[Remark by alexk: see #852]

[olexandr-konovalov]

Thanks @ChristopherRussell - in the meantime, I am going to merge PR #851 and start to work on organising the Primitive Groups Library into a GAP package in https://github.com/gap-packages/primgrp. Hopefully when you will be back you will be able to contribute to primitive groups library inside the package instead of the main repository.