Restore "huge" package type

[mkoeppe]

(from #29901)

... for packages like database_stein_watkins and polytopes_db_4d

Parts of sage_bootstrap know about this package type, but other scripts don't.

Our GH Actions scripts filter out these packages by name. This can be refactored.

With the present ticket, we can now say sage --package list :optional: :experimental: --no-file huge to filter out the huge packages.

Huge packages are now marked by the presence of the file huge; so this is actually orthogonal to the package type.

We also add files named has_nonfree_dependencies to some other packages so that they can be filtered out by the same mechanism.

CC: @jhpalmieri @dimpase @orlitzky @seblabbe @Etn40ff @tobiasdiez

Component: build

Author: Matthias Koeppe

Branch/Commit: 15cc20c

Reviewer: John Palmieri

Issue created by migration from https://trac.sagemath.org/ticket/29919

[mkoeppe]

comment:5

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.

[mkoeppe]

Author: Matthias Koeppe

[mkoeppe]

Branch: u/mkoeppe/restore__huge__package_type

[mkoeppe]

Changed dependencies from #29901, #20104 to none

[mkoeppe]

Commit: ce1924c

[mkoeppe]

New commits:

e6b82e8	build/sage_bootstrap: Remove :huge: package class
74ca883	sage -package list: Add option --no-file
ce1924c	build/pkgs/{database_stein_watkins, polytopes_db_4d}: Mark as huge packages

[sagetrac-git]

Changed commit from ce1924c to 554ad59

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

554ad59

build/pkgs/{scipoptsuite,sage_numerical_backends_cplex,sage_numerical_backends_gurobi}/has_nonfree_dependencies: New

[jhpalmieri]

comment:14

Can you document both huge and has_nonfree_dependencies in the developer's guide?

[mkoeppe]

comment:15

Good idea

[orlitzky]

comment:16

I think we should have a cleaner way to add "tags" to packages, but in lieu of general solution this gets the job done.

The nonfree-dependencies thing is a bit of a legal mess. Users who install such a package lose the right to distribute their copy of sage. Entities such as universities may not want to wade into that sort of grey area. Anyway, that's orthogonal to this ticket.

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

b401649

src/doc/en/developer/packaging.rst: Explain package tags has_nonfree_dependencies, huge

[sagetrac-git]

Changed commit from 554ad59 to b401649

[sagetrac-git]

Changed commit from b401649 to bf0c923

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

bf0c923

src/doc/en/developer/packaging.rst: Add section title

[sagetrac-git]

Changed commit from bf0c923 to 15cc20c

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

15cc20c

src/doc/en/developer/packaging.rst: Typo

[jhpalmieri]

comment:20

Would

if any(not pkg.has_file(filename) for filename in filenames):

be more efficient than the current

if not all(pkg.has_file(filename) for filename in filenames):

and similarly a few lines later with not any(...)? Or does Python do this sort of optimization automatically?

[jhpalmieri]

comment:21

Never mind, I guess it doesn't matter.

[mkoeppe]

comment:22

There's no difference in efficiency - both any and all shortcircuit.
And I think there is no difference in clarity

[jhpalmieri]

comment:23

I got confused and thought that the two might short-circuit at different points, but they will short-circuit at exactly the same point, so it makes no difference.

This is not your fault and should not be fixed on this ticket, but I wish that the code in expand_class.py were documented. Anyway, let's move forward with this.

[jhpalmieri]

Reviewer: John Palmieri

[mkoeppe]

comment:24

Thanks!

[vbraun]

Changed branch from u/mkoeppe/restore__huge__package_type to 15cc20c