Rename install-requires.txt to version_requirements.txt

[mkoeppe]

As discussed in #36982:

• the name "install-requires" has become outdated with the transition to the new names set by the pyproject.toml format (see https://setuptools.pypa.io/en/latest/userguide/dependency_management.html#declaring-required-dependency, compare the tabs "pyproject.toml" vs. "setup.cfg")
• this will help with Complete our GitHub dependency graph (javascript help needed) #35890, as GitHub will be able to just read the version_requirements.txt files

#36982 (comment)

Notes for reviewers:

• This is a simple, limited-scope improvement and not intended as a long-term commitment to the format of the files in build/pkgs/....
• PRs Refactor bootstrap, bootstrap-conda, m4/sage_spkg_collect.m4, sage-spkg-info through sage --package properties, sage-get-system-packages #37430, sage -package dependencies #37350, Refactor bootstrap-conda, m4/sage_spkg_collect.m4 through sage-package; handle dependencies_build #36740 remove direct access to build/pkgs in favor of using the sage_bootstrap API (sage --package).

📝 Checklist

• The title is concise, informative, and self-explanatory.
• The description explains in detail what this PR is about.
• I have linked a relevant issue or discussion.
• I have created tests covering the changes.
• I have updated the documentation accordingly.

⌛ Dependencies

• Depends on Developer Guide: Improvements to explanation of version files #37401

[kwankyu]

• the name "install-requires" has become outdated with the transition to the new names set by the pyproject.toml format (see https://setuptools.pypa.io/en/latest/userguide/dependency_management.html#declaring-required-dependency, compare the tabs "pyproject.toml" vs. "setup.cfg")

There it is "dependencies"?

[mkoeppe]

That's right. The term "install-requires" has been eliminated.

[kwankyu]

Then you made up the name "version_requirements.txt", instead of using "dependencies.txt"?

[mkoeppe]

That's right.

[kwankyu]

(we may come up with a name better than both, but I have no idea now...)

[mkoeppe]

it needs to end with "_requirements.txt" so that GitHub finds it (see link in PR description). I didn't have a better idea what to stick in front.

[kwankyu]

Simply "install_requirements.txt"?

[mkoeppe]

For inspiration: https://packaging.python.org/en/latest/specifications/dependency-specifiers/

[kwankyu]

I cannot come up with an obviously better name than "version_requirements.txt". This file is about version requirements on dependencies (pip packages on pypi). So other possible names are "dependency_requirements.txt" or "pip_requirements.txt". No strong opinion anyway...

[dimpase]

I still don't get why we cannot just use requirements.txt.

Same format, essentially the same purpose, ease of conversion to pip package.

If there is any need to prepend foo_ to this name it can be done on the fly by tools.

[kwankyu]

Perhaps because an spkg is not a python package?

[kwankyu]

Another suggestion: "package_requirements".

[dimpase]

Perhaps because an spkg is not a python package?

These are Python packages we are talking about, these which are vendored by Sage, i.e. made into spkg packaged as tarfiles or wheels. Package names in install-requires.txt are not spkg names, these are Python package names.
You can rename install-requires.txt to requirements.txt, remove package_version.txt and checksums.ini, and usually
such a package can still be installed, as a PyPI package, by-passing Sage's tarballs/wheels.

If it's a standard package, it's an illegal operation under the current policy; as you know I called a sage-devel vote on changing this policy...

[kwankyu]

I agree that "requirements.txt" seems the best choice. According to the developer guide, presence of "requirements.txt" is used to differentiate "pip" spkg from "normal" and "wheel" spkg. How about using presence of "checksums.ini" for this purpose? Do I miss something? Is there other reason that "requirements.txt" cannot be a choice?

[mkoeppe]

I still don't get why we cannot just use requirements.txt.

@dimpase I explained it to you in #37401 (comment); complete the review there, please

[mkoeppe]

According to the developer guide, presence of "requirements.txt" is used to differentiate "pip" spkg from "normal" and "wheel" spkg.

Yes, that's right.

How about using presence of "checksums.ini" for this purpose?

This could be done (best after the refactoring in #36740, which needs review), but I would still not use requirements.txt for the acceptable version ranges of normal/wheel packages.

Edit: The part of #36740 needed for that is now split out as #37430 (needs review)

[kwankyu]

Because requirements.txt means something else. It is the name one uses for version pinnings, as in pip install -r requirements.txt.

But you can use version ranges too in the file.

install-requires is the traditional name (in setuptools) for declaring dependencies.

Yes. But as far as I understand, install-requires in an spkg specifies the "dependency" of the spkg, which is the pip package that the spkg installs. The "dependency" can be well described by the "requirements.txt" file. (the spkg "requires" the specified versions.)

Anyway, "version-requirements.txt" is slightly different from "requirements.txt". I feel that "version" prefix emphasizes the version too much: "the spkg requires the version(?)"...

[mkoeppe]

Because requirements.txt means something else. It is the name one uses for version pinnings, as in pip install -r requirements.txt.

But you can use version ranges too in the file.

I should have written: requirements.txt ... is the file name one uses for version pinnings, as in pip install -r requirements.txt.

[mkoeppe]

In Python packaging, one distinguishes "abstract dependencies" (install-requires) vs "concrete dependencies" (requirements.txt). See e.g. https://martin-thoma.com/python-requirements/#abstract-dependencies

That's why using the name requirements.txt for declaring our acceptable version ranges would be bad -- it would confuse anyone who is familiar with Python packaging practices.

[mkoeppe]

install-requires is the traditional name (in setuptools) for declaring dependencies.

Yes. But as far as I understand, install-requires in an spkg specifies the "dependency" of the spkg, which is the pip package that the spkg installs.

No, that's not a good way of viewing the relation of the spkg and the pip package. The spkg does not "depend" on it. It "represents" it.

I chose the name "install-requires.txt" to mean: If a Python distribution needs to declare this package as an "install-requires", use the contents of this file for it. And that's what is done when we create pkgs/*/setup.cfg from the corresponding setup.cfg.m4.

By the way, we are also generating "pkgs/*/requirements.txt" files such as pkgs/sagemath-standard/requirements.txt. The versions in there are pinned (using the information from package-version.txt files).

[kwankyu]

In Python packaging, one distinguishes "abstract dependencies" (install-requires) vs "concrete dependencies" (requirements.txt). See e.g. https://martin-thoma.com/python-requirements/#abstract-dependencies

That's why using the name requirements.txt for declaring our acceptable version ranges would be bad -- it would confuse anyone who is familiar with Python packaging practices.

Thanks for the read. I see now why you reject the name requirements.txt: In python packagaing, install-requires and requirements.txt have (loosely) separate uses.

I chose the name "install-requires.txt" to mean: If a Python distribution needs to declare this package as an "install-requires", use the contents of this file for it. And that's what is done when we create pkgs/*/setup.cfg from the corresponding setup.cfg.m4.

This explains why I got confused. The name "install-requires.txt" cannot be explained within spkg context. You see the spkg as a "dependency" of the Python distribution, and the content of install-requires.txt is supposed to be used by the Python distribution. I was thinking within spkg context.

Now the Python distribution uses pyproject.toml, and the file uses the name "dependencies" instead of old "install-requires". Then logically, you want to replace "install-requires.txt" with "dependencies.txt", but "dependencies" is already being used for spkg dependencies. Hence we need a new name. Am I right?

[mkoeppe]

logically, you wan to replace "install-requires.txt" with "dependencies.txt", but "dependencies" is already being used for spkg dependencies. Hence we need a new name. Am I right?

Exactly that. "install-requires" was an idiosyncratic name only used for one purpose in Python packaging, and hence it was unambiguous. "dependencies" is not just already used, it's also too generic.

[jhpalmieri]

I'm posting to record a vote of -1 on behalf of Tobias Diez.

[kiwifb]

It seems to me the dispute is really about the naming. In any case we need to move from the current name +1 for me.

[kwankyu]

Still +1 from me. version_requirement.txt is to replace install-requires.txt, which is different from requirement.txt.

[NathanDunfield]

Looks good to me. I agree @kwankyu in #36999 that version_requirements.txt is the best choice of name here.

[roed314]

Based on @kwankyu's suggestion and discussion by the new Code of Conduct Committee, I'm setting this ticket to Needs Review so that all the disputed tickets start with a clean slate prior to being update based on voting. Starting at midnight US/Pacific time tonight anyone involved should feel free to set the status to positive review or needs review in accordance with the new policy.

[NathanDunfield]

Per the comment by @roed314, I have set to positive review counting:

+1: @mkoeppe, @kwankyu, @kiwifb, and myself
-1: @tobiasdiez, @dimpase

[vbraun]

Please try building from a clean checkout, I'm getting lots of sage-spkg-info: unknown package xz for all packages

[github-actions]

Documentation preview for this PR (built with commit 918448a; changes) is ready! 🎉