Replace bootstrap-conda by grayskull and update conda lock files #37447
Conversation
mkoeppe
added
the
disputed
dimpase
removed
the
disputed
|
I've removed |
|
do you mean Anyhow, after I do the latter, I get What do I do wrong? Does it need to be run in a conda env? |
mkoeppe
added
the
disputed
Sorry, forgot that the script uses conda-lock as well. Please try again after |
dimpase
removed
the
disputed
mkoeppe
added
the
disputed
|
Why do I see on an amd64 box while running |
|
I was getting errors about unresolved packages,so I applied diff --git a/tools/update-conda.py b/tools/update-conda.py
index f123c7e4151..176b016d183 100644
--- a/tools/update-conda.py
+++ b/tools/update-conda.py
@@ -20,10 +20,6 @@ options = parser.parse_args()
platforms = {
"linux-64": "linux",
- "linux-aarch64": "linux-aarch64",
- "osx-64": "macos",
- "osx-arm64": "macos-arm64",
- "win-64": "win",
}
pythons = ["3.9", "3.10", "3.11"]
tags = ["", "-dev"]and this worked. Obv. |
Note that the
Good catch. Thanks! I've played around with the windows packages for a bit, but then forgot to uncomment it again. Done now. |
|
Please clarify what about this PR is disputed. (@tobiasdiez is it possible that @mkoeppe is lacking the permission to comment here?) |
|
I took the liberty to add the "needs work" label since it's unclear what's disputed here. (Also, you could maybe check or remove the checklist at the top of the PR to make it clear that there's nothing from that checklist missing here.) |
This comment was marked as abuse.
This comment was marked as abuse.
|
@saraedum @roed314 @jhpalmieri I set "disputed" here because when the PR was opened, I was not able to comment because Tobias had blocked me without explanation or giving notice to the sage-abuse committee. It appears that now I can comment, so I am removing "disputed". There has not been any meaningful review yet here, so conducting a vote would seem premature. |
mkoeppe
removed
the
disputed
|
It is correct that on my own PRs, @dimpase is blocked because of his persistent and aggressive code of conduct violations, including the type of denunciations that he has just written above. |
|
In any case, this PR here is based on #37446, which is in disputed status because of #37446 (comment) |
This comment was marked as abuse.
This comment was marked as abuse.
|
Documentation preview for this PR (built with commit 629363a; changes) is ready! 🎉 |
There was a problem hiding this comment.
I understand that there are reservations about having the pyproject.toml in the root directory. Otherwise, this PR does seem like a reasonable idea to me (replace a shell script with Python and relay some of the heavy lifting to an external project.)
Is there a better place for the pyproject.toml that would make this work for everybody?
| @@ -0,0 +1,154 @@ | |||
| #!/usr/bin/env python3 | |||
There was a problem hiding this comment.
I think it would be good to add an explanation at the top that explains what this script does. Or maybe add a --help to the argument parser that prints such documentation.
There was a problem hiding this comment.
I've now added a bit of documentation in the README file in the same directory, and a very short comment on the usage in this file. Do you think this is sufficient?
tools/update-conda.py
Outdated
| return dev_dependencies | ||
|
|
||
|
|
||
| update_conda(Path(options.sourcedir)) |
There was a problem hiding this comment.
Not super important but if we put all the global bits of this script in a def main() and do the usual if __name__ == "__main__" or similar, then this could be imported by others to use bits of the machinery for other purposes.
|
Creating a new top-level And actually, there's no need for a new top-level directory: We already have a place for scripts of exactly this kind -- it is |
|
Thanks Dima! |
tobiasdiez
force-pushed
the
conda-grayskull
branch
from
6a498f7 to
18b4fc9
Compare
|
<!-- ^ Please provide a concise and informative title. --> <!-- ^ Don't put issue numbers in the title, do this in the PR description below. --> <!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method to calculate 1 + 2". --> <!-- v Describe your changes below in detail. --> <!-- v Why is this change required? What problem does it solve? --> <!-- v If this PR resolves an open issue, please link to it here. For example, "Fixes sagemath#12345". --> Newer versions of numpy use meson instead of the old distutils. Needed for sagemath#37447. ### 📝 Checklist <!-- Put an `x` in all the boxes that apply. --> - [ ] The title is concise and informative. - [ ] The description explains in detail what this PR is about. - [ ] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation and checked the documentation preview. ### ⌛ Dependencies <!-- List all open PRs that this PR logically depends on. For example, --> <!-- - sagemath#12345: short description why this is a dependency --> <!-- - sagemath#34567: ... --> URL: sagemath#38983 Reported by: Tobias Diez Reviewer(s): Dima Pasechnik
|
How about something like this? (it does fix --- a/bootstrap
+++ b/bootstrap
@@ -225,14 +225,14 @@ save () {
build/make/Makefile-auto.in \
src/doc/en/installation/*.txt \
$(find src/doc/en/reference/spkg -name index.rst -prune -o -maxdepth 1 -name "*.rst" -print) \
- environment-3.[89].yml environment-3.1[0-9].yml \
- src/Pipfile \
+ environment-3.[9]-*.yml environment-3.1[0-9]-*.yml \
build/pkgs/cypari/version_requirements.txt \
build/pkgs/cysignals/version_requirements.txt \
build/pkgs/cython/version_requirements.txt \
build/pkgs/gmpy2/version_requirements.txt \
build/pkgs/jupyter_core/version_requirements.txt \
build/pkgs/memory_allocator/version_requirements.txt \
+ build/pkgs/meson/version_requirements.txt \
build/pkgs/numpy/version_requirements.txt \
build/pkgs/pkgconfig/version_requirements.txt \
build/pkgs/pplpy/version_requirements.txt \Also, what's the point of |
github-actions
bot
added
s: needs review
and removed
s: positive review
s: needs work
labels
<!-- ^ Please provide a concise and informative title. --> <!-- ^ Don't put issue numbers in the title, do this in the PR description below. --> <!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method to calculate 1 + 2". --> <!-- v Describe your changes below in detail. --> <!-- v Why is this change required? What problem does it solve? --> <!-- v If this PR resolves an open issue, please link to it here. For example, "Fixes sagemath#12345". --> Newer versions of numpy use meson instead of the old distutils. Needed for sagemath#37447. ### 📝 Checklist <!-- Put an `x` in all the boxes that apply. --> - [ ] The title is concise and informative. - [ ] The description explains in detail what this PR is about. - [ ] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation and checked the documentation preview. ### ⌛ Dependencies <!-- List all open PRs that this PR logically depends on. For example, --> <!-- - sagemath#12345: short description why this is a dependency --> <!-- - sagemath#34567: ... --> URL: sagemath#38983 Reported by: Tobias Diez Reviewer(s): Dima Pasechnik
<!-- ^ Please provide a concise and informative title. --> <!-- ^ Don't put issue numbers in the title, do this in the PR description below. --> <!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method to calculate 1 + 2". --> <!-- v Describe your changes below in detail. --> <!-- v Why is this change required? What problem does it solve? --> <!-- v If this PR resolves an open issue, please link to it here. For example, "Fixes sagemath#12345". --> Newer versions of numpy use meson instead of the old distutils. Needed for sagemath#37447. ### 📝 Checklist <!-- Put an `x` in all the boxes that apply. --> - [ ] The title is concise and informative. - [ ] The description explains in detail what this PR is about. - [ ] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation and checked the documentation preview. ### ⌛ Dependencies <!-- List all open PRs that this PR logically depends on. For example, --> <!-- - sagemath#12345: short description why this is a dependency --> <!-- - sagemath#34567: ... --> URL: sagemath#38983 Reported by: Tobias Diez Reviewer(s): Dima Pasechnik
Thanks!
These files have been deleted now in #39031. |
tobiasdiez
force-pushed
the
conda-grayskull
branch
from
629363a to
c1f8dbc
Compare
…da lock files
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->
Using the metadata added in sagemath#37446,
we automatically generate the conda environment files. This no longer
makes any reference to the `conda.txt` files contained in sage-the-
distribution. Thus sage-on-top-of-conda is now completely independent of
sage-the-distribution (only relying on information specified by sage-
the-library). In particular, after this PR is merged the `conda.txt`
files could be deleted from sage-the-distribution.
In particular, we no longer need to maintain the mapping of pypi
packages to conda packages but instead can rely on the offical mappings
maintained by the conda team (https://github.com/regro/cf-graph-
countyfair/tree/master/mappings/pypi).
To test:
```
pip install grayskull conda-lock
python tools/update-conda.py
```
The updated conda files are committed here as well.
Moreover, the `environment-dev` files have been deleted as there was
only little difference between these files and some people rightfully
complained that having too many files in the root folder is distracting.
As a byproduct, this fixes sagemath#34626.
<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
<!-- If your change requires a documentation PR, please link it
appropriately. -->
### 📝 Checklist
<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->
- [ ] The title is concise, informative, and self-explanatory.
- [ ] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.
### ⌛ Dependencies
- sagemath#37446: specifies additional
requirements in the pyproject.toml which are used to generate the conda
env files
- sagemath#38983: for the update of numpy
- sagemath#38982: to fix meson build
<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
URL: sagemath#37447
Reported by: Tobias Diez
Reviewer(s): Dima Pasechnik, Julian Rüth, Tobias Diez
…da lock files
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->
Using the metadata added in sagemath#37446,
we automatically generate the conda environment files. This no longer
makes any reference to the `conda.txt` files contained in sage-the-
distribution. Thus sage-on-top-of-conda is now completely independent of
sage-the-distribution (only relying on information specified by sage-
the-library). In particular, after this PR is merged the `conda.txt`
files could be deleted from sage-the-distribution.
In particular, we no longer need to maintain the mapping of pypi
packages to conda packages but instead can rely on the offical mappings
maintained by the conda team (https://github.com/regro/cf-graph-
countyfair/tree/master/mappings/pypi).
To test:
```
pip install grayskull conda-lock
python tools/update-conda.py
```
The updated conda files are committed here as well.
Moreover, the `environment-dev` files have been deleted as there was
only little difference between these files and some people rightfully
complained that having too many files in the root folder is distracting.
As a byproduct, this fixes sagemath#34626.
<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
<!-- If your change requires a documentation PR, please link it
appropriately. -->
### 📝 Checklist
<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->
- [ ] The title is concise, informative, and self-explanatory.
- [ ] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.
### ⌛ Dependencies
- sagemath#37446: specifies additional
requirements in the pyproject.toml which are used to generate the conda
env files
- sagemath#38983: for the update of numpy
- sagemath#38982: to fix meson build
<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
URL: sagemath#37447
Reported by: Tobias Diez
Reviewer(s): Dima Pasechnik, Julian Rüth, Tobias Diez
Using the metadata added in #37446, we automatically generate the conda environment files. This no longer makes any reference to the
conda.txtfiles contained in sage-the-distribution. Thus sage-on-top-of-conda is now completely independent of sage-the-distribution (only relying on information specified by sage-the-library). In particular, after this PR is merged theconda.txtfiles could be deleted from sage-the-distribution.In particular, we no longer need to maintain the mapping of pypi packages to conda packages but instead can rely on the offical mappings maintained by the conda team (https://github.com/regro/cf-graph-countyfair/tree/master/mappings/pypi).
To test:
The updated conda files are committed here as well.
Moreover, the
environment-devfiles have been deleted as there was only little difference between these files and some people rightfully complained that having too many files in the root folder is distracting.As a byproduct, this fixes #34626.
📝 Checklist
⌛ Dependencies
pyproject.toml#37446: specifies additional requirements in the pyproject.toml which are used to generate the conda env files