Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v1.20241002 #50

Open
wants to merge 29 commits into
base: main
Choose a base branch
from
Open

v1.20241002 #50

wants to merge 29 commits into from

Conversation

charman2
Copy link
Contributor

@charman2 charman2 commented Oct 2, 2024

Checklist

  • Used a personal fork of the feedstock to propose changes
  • Bumped the build number (if the version is unchanged)
  • Reset the build number to 0 (if the version changed)
  • Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
  • Ensured the license file is being packaged.

regro-cf-autotick-bot and others added 3 commits June 16, 2024 02:31
TL;DR: The way we build against numpy has changed as of numpy 2.0. This bot
PR has updated the recipe to account for the changes (see below for details).
The numpy 2.0 package itself is currently only available from a special release
channel (`conda-forge/label/numpy_rc`) and will not be available on the main
`conda-forge` channel until the release of numpy 2.0 GA.

The biggest change is that we no longer need to use the oldest available numpy
version at build time in order to support old numpy version at runtime - numpy
will by default use a compatible ABI for the oldest still-supported numpy versions.

Additionally, we no longer need to use `{{ pin_compatible("numpy") }}` as a
run requirement - this has been handled for more than two years now by a
run-export on the numpy package itself. The migrator will therefore remove
any occurrences of this.

However, by default, building against numpy 2.0 will assume that the package
is compatible with numpy 2.0, which is not necessarily the case. You should
check that the upstream package explicitly supports numpy 2.0, otherwise you
need to add a `- numpy <2.0dev0` run requirement until that happens (check numpy
issue 26191 for an overview of the most important packages).

Note that the numpy release candidate promises to be ABI-compatible with the
final 2.0 release. This means that building against 2.0.0rc1 produces packages
that can be published to our main channels.

If you already want to use the numpy 2.0 release candidate yourself, you can do
```
conda config --add channels conda-forge/label/numpy_rc
```
or add this channel to your `.condarc` file directly.

### To-Dos:
  * [ ] Match run-requirements for numpy (i.e. check upstream `pyproject.toml` or however the project specifies numpy compatibility)
    * If upstream is not yet compatible with numpy 2.0, add `numpy <2.0dev0` upper bound under `run:`.
    * If upstream is already compatible with numpy 2.0, nothing else should be necessary in most cases.
    * If upstream requires a minimum numpy version newer than 1.19, you can add `numpy >=x.y` under `run:`.
  * [ ] Remove any remaining occurrences of `{{ pin_compatible("numpy") }}` that the bot may have missed.

PS. If the build does not compile anymore, this is almost certainly a sign that
the upstream project is not yet ready for numpy 2.0; do not close this PR until
a version compatible with numpy 2.0 has been released upstream and on this
feedstock (in the meantime, you can keep the bot from reopening this PR in
case of git conflicts by marking it as a draft).
@charman2
Copy link
Contributor Author

charman2 commented Oct 2, 2024

@ conda-forge-admin, please rerender

@charman2
Copy link
Contributor Author

charman2 commented Oct 2, 2024

@conda-forge-admin, please rerender

@conda-forge-admin
Copy link
Contributor

conda-forge-admin commented Oct 2, 2024

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found some lint.

Here's what I've got...

For recipe/meta.yaml:

  • Selectors are suggested to take a <two spaces>#<one space>[<expression>] form. See lines [29, 30, 31, 37]

@conda-forge-admin
Copy link
Contributor

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you but ran into some issues. Please check the output logs of the GitHub actions workflow below for more details. You can also ping conda-forge/core for further assistance or you can try rerendering locally.

The following suggestions might help debug any issues:

  • Is the recipe/{meta.yaml,recipe.yaml} file valid?
  • If there is a recipe/conda-build-config.yaml file in the feedstock make sure that it is compatible with the current global pinnnings.
  • Is the fork used for this PR on an organization or user GitHub account? Automated rerendering via the webservices admin bot only works for user GitHub accounts.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/11151186572.

@charman2
Copy link
Contributor Author

charman2 commented Oct 2, 2024

@conda-forge-admin, please rerender

@conda-forge-admin
Copy link
Contributor

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you but ran into some issues. Please check the output logs of the GitHub actions workflow below for more details. You can also ping conda-forge/core for further assistance or you can try rerendering locally.

The following suggestions might help debug any issues:

  • Is the recipe/{meta.yaml,recipe.yaml} file valid?
  • If there is a recipe/conda-build-config.yaml file in the feedstock make sure that it is compatible with the current global pinnnings.
  • Is the fork used for this PR on an organization or user GitHub account? Automated rerendering via the webservices admin bot only works for user GitHub accounts.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/11151267366.

Ciaran Harman added 2 commits October 2, 2024 16:42
@charman2
Copy link
Contributor Author

charman2 commented Oct 2, 2024

@conda-forge-admin, please rerender

@conda-forge-admin
Copy link
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

@conda-forge-admin
Copy link
Contributor

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you but ran into some issues. Please check the output logs of the GitHub actions workflow below for more details. You can also ping conda-forge/core for further assistance or you can try rerendering locally.

The following suggestions might help debug any issues:

  • Is the recipe/{meta.yaml,recipe.yaml} file valid?
  • If there is a recipe/conda-build-config.yaml file in the feedstock make sure that it is compatible with the current global pinnnings.
  • Is the fork used for this PR on an organization or user GitHub account? Automated rerendering via the webservices admin bot only works for user GitHub accounts.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/11151362628.

Ciaran Harman added 2 commits October 2, 2024 16:50
Copy link
Member

@jakirkham jakirkham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Ciaran! 🙏

Added a few suggestions below

In particular think the NumPy pins may be causing us issues in getting CI to work

recipe/meta.yaml Outdated Show resolved Hide resolved
recipe/meta.yaml Outdated Show resolved Hide resolved
recipe/meta.yaml Outdated Show resolved Hide resolved
host:
- python
- setuptools
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to confirm, there is no longer any use of setuptools in the build?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like CI is running into errors not finding distutils, which setuptools vendors

Can we try adding this back?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No harm in trying, but it shouldn't need it. The main fix in the new version of the package was to migrate from setuptools to meson, since distutils is deprecated

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For clarity this is the traceback from CI:

  Traceback (most recent call last):
    File "/home/conda/feedstock_root/build_artifacts/mesas_1727984139908/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/bin/f2py", line 10, in <module>
      sys.exit(main())
    File "/home/conda/feedstock_root/build_artifacts/mesas_1727984139908/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/python3.10/site-packages/numpy/f2py/f2py2e.py", line 691, in main
      run_compile()
    File "/home/conda/feedstock_root/build_artifacts/mesas_1727984139908/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/python3.10/site-packages/numpy/f2py/f2py2e.py", line 619, in run_compile
      from numpy.distutils.system_info import get_info
    File "/home/conda/feedstock_root/build_artifacts/mesas_1727984139908/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/python3.10/site-packages/numpy/distutils/system_info.py", line 196, in <module>
      from numpy.distutils.command.config import config as cmd_config
    File "/home/conda/feedstock_root/build_artifacts/mesas_1727984139908/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/python3.10/site-packages/numpy/distutils/command/config.py", line 19, in <module>
      from numpy.distutils.mingw32ccompiler import generate_manifest
    File "/home/conda/feedstock_root/build_artifacts/mesas_1727984139908/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/python3.10/site-packages/numpy/distutils/mingw32ccompiler.py", line 28, in <module>
      from distutils.msvccompiler import get_build_version as get_build_msvc_version
  ModuleNotFoundError: No module named 'distutils.msvccompiler'

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this issue traces back to f2py ( numpy/numpy#24838 ), which was fixed in NumPy 2 ( numpy/numpy#25123 )

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have merged the NumPy 2 migrator from PR ( #49 ) into this PR. Let's see if that helps

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In some jobs this appears to be fixed! 🎉 However there are new errors (noted below) 😞 Please take a look at the comments below

We still see this error in a few jobs. So let's keep this thread open

Though setuptools isn't helping in any case. So think we can go ahead and drop it again

recipe/meta.yaml Show resolved Hide resolved
recipe/meta.yaml Outdated Show resolved Hide resolved
recipe/meta.yaml Outdated Show resolved Hide resolved
recipe/meta.yaml Outdated Show resolved Hide resolved
@jakirkham
Copy link
Member

@conda-forge-admin , please re-render

conda-forge-webservices[bot] and others added 3 commits October 3, 2024 19:33
recipe/meta.yaml Outdated Show resolved Hide resolved
@jakirkham
Copy link
Member

In a few CI jobs (like this one), we are seeing this error

import: 'mesas'
import: 'mesas.sas.solve'
Traceback (most recent call last):
  File "/Users/runner/miniforge3/conda-bld/mesas_1728001177938/test_tmp/run_test.py", line 5, in <module>
    import mesas.sas.solve
ModuleNotFoundError: No module named 'mesas.sas.solve'

Do we need to update these tests for this version?

test:
imports:
- mesas
- mesas.sas.solve

@jakirkham
Copy link
Member

The macOS ARM builds, which use cross-compilation, are having issues running. Have seen similar issues with Meson before. Think we need to adapt the build logic here for them

https://github.com/conda-forge/pywavelets-feedstock/blob/2bc11d63d4c63ca91f31a3247f8282b2bfae0e76/recipe/meta.yaml#L17-L22

Note this approach uses python-build, which we would need to add to requirements/host

recipe/meta.yaml Outdated Show resolved Hide resolved
@jakirkham
Copy link
Member

Windows Fortran builds have this error

  [6/6] Linking target mesas_solve.cp312-win_amd64.pyd
  FAILED: mesas_solve.cp312-win_amd64.pyd
  "link"  /MACHINE:x64 /OUT:mesas_solve.cp312-win_amd64.pyd mesas_solve.cp312-win_amd64.pyd.p/solve.f90.obj mesas_solve.cp312-win_amd64.pyd.p/mesas_solvemodule.c.obj mesas_solve.cp312-win_amd64.pyd.p/mesas_solve-f2pywrappers2.f90.obj mesas_solve.cp312-win_amd64.pyd.p/7c6f89aff64b5d6ae9af36457a578bcaa22407a2_.._.._f2py_src_fortranobject.c.obj "/LIBPATH:D:/bld/mesas_1728001287970/_build_env/Library/mingw-w64/lib/gcc/x86_64-w64-mingw32/5.3.0" "/LIBPATH:D:/bld/mesas_1728001287970/_build_env/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0" "/LIBPATH:D:/bld/mesas_1728001287970/_build_env/Library/mingw-w64/lib/gcc" "/LIBPATH:D:/bld/mesas_1728001287970/_build_env/Library/mingw-w64/bin/../lib/gcc" "/LIBPATH:D:/bld/mesas_1728001287970/_build_env/Library/mingw-w64/x86_64-w64-mingw32/lib" "/LIBPATH:D:/bld/mesas_1728001287970/_build_env/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/lib/../lib" "/LIBPATH:D:/bld/mesas_1728001287970/_build_env/Library/mingw-w64/lib" "/LIBPATH:D:/bld/mesas_1728001287970/_build_env/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../lib" "/LIBPATH:D:/bld/mesas_1728001287970/_build_env/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/lib" "/LIBPATH:D:/bld/mesas_1728001287970/_build_env/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../.." "/release" "/nologo" "/OPT:REF" "/DLL" "/IMPLIB:mesas_solve.cp312-win_amd64.lib" "D:\bld\mesas_1728001287970\_h_env\libs\python312.lib" "quadmath.lib" "kernel32.lib" "user32.lib" "gdi32.lib" "winspool.lib" "shell32.lib" "ole32.lib" "oleaut32.lib" "uuid.lib" "comdlg32.lib" "advapi32.lib" "gfortran.lib"
  LINK : fatal error LNK1181: cannot open input file 'quadmath.lib'

This suggests either the library search path needs a fix or quadmath is missing

More likely it is the former, but let's try to solve a few of the other issues before digging into this one

@charman2
Copy link
Contributor Author

charman2 commented Oct 4, 2024

mesas.sas.solve is the fortran number-crunching code that the python is wrapping. The import test should work. If it is being built but the test is failing I guess the extension is being put in the wrong place?

I'll try that script for the mac ARM builds

No idea what the deal is with the quadmath library

Again, I appreciate the help. I'm starting to tear my hair out!

@charman2
Copy link
Contributor Author

charman2 commented Oct 4, 2024

My attempt at the script failed with

conda_build.exceptions.CondaBuildException: Found a build.sh script and a build/script section inside meta.yaml. Either remove the build.sh script or remove the build/script section in meta.yaml.

across the board

@jakirkham
Copy link
Member

Yes this would need to either...

  • ...go in the existing build.sh and bld.bat scripts
  • ...or those scripts would need to be removed and the scripts section used instead

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants