SciPy v1.15.0

[h-vetinari]

For conda-forge/conda-smithy#2060 & scipy/scipy#21553

scipy/scipy#21561

Testing for 1.15.0rc1

[conda-forge-webservices]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

[h-vetinari]

Raised an issue for 5eb8df4 / 36a062e: conda-forge/conda-smithy#2061

[h-vetinari]

I cannot tell exactly what's going wrong here @ilayn, but note that we make it to the very last of the

import tests

scipy-feedstock/recipe/meta.yaml

Lines 130 to 172 in 07e1f9f

	imports:
	- scipy
	# reference for public API is effectively
	# https://github.com/scipy/scipy/blob/main/scipy/_lib/tests/test_public_api.py
	- scipy.cluster
	- scipy.cluster.hierarchy
	- scipy.cluster.vq
	- scipy.constants
	- scipy.datasets
	- scipy.fft
	- scipy.fftpack
	- scipy.integrate
	- scipy.interpolate
	- scipy.io
	- scipy.io.arff
	- scipy.io.matlab
	- scipy.io.wavfile
	- scipy.linalg
	- scipy.linalg.blas
	- scipy.linalg.cython_blas
	- scipy.linalg.cython_lapack
	- scipy.linalg.interpolative
	- scipy.linalg.lapack
	- scipy.misc
	- scipy.ndimage
	- scipy.odr
	- scipy.optimize
	- scipy.signal
	- scipy.signal.windows
	- scipy.sparse
	- scipy.sparse.csgraph
	- scipy.sparse.linalg
	- scipy.spatial
	- scipy.spatial.distance
	- scipy.spatial.transform
	- scipy.special
	- scipy.stats
	- scipy.stats.contingency
	- scipy.stats.distributions
	- scipy.stats.mstats
	- scipy.stats.qmc
	- scipy.stats.sampling
	commands:

In fact, we even do so twice (well, conda-build does a weird log duplication for the import tests; not 100% sure it's actually run twice). In any case, it seems like something is leaving an error code (or some other dirtied resource) behind so that conda-build sees some kind of failure at the end of the import tests. It's clear as mud to me what might be the cause.

[ilayn]

Whoa, that's really cryptic stuff. Regardless of the issue hats off to you for handling this chaos. I can only see one tiny sliver of hope that the failure in the azure pipelines stacktrace ends with

conda_build.exceptions.CondaBuildUserError: TESTS FAILED: scipy-1.14.1-py313hdf33807_0.conda

https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=1028042&view=logs&j=00f5923e-fdef-5026-5091-0d5a0b3d5a2c&t=f42c1d49-d161-5453-1c0e-990f827a8536&l=10989

Also the scipy version a bit above in the package list says SciPy version 1.14.1 and local. So maybe a cache needs to be invalidated?

https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=1028042&view=logs&j=00f5923e-fdef-5026-5091-0d5a0b3d5a2c&t=f42c1d49-d161-5453-1c0e-990f827a8536&l=10859

If it's not that, I'm with you with the snorkel view about this.

[h-vetinari]

Unfortunately you're fishing for red herrings here. 😅

That's a regular message for failing tests, and the local is important because it means we're testing the just-built scipy (rather than an already-published one). Finally, the version is reported as old because I haven't updated the conda-side metadata in this PR.

[h-vetinari]

But it is indeed unusual to fail already at the import stage without any other info. During the pytest stage we normally get better errors. I guess that's also an option here - disable the import tests and see what happens...

[ilayn]

I think this is not scipy related but something else is broken in the toolchain. Looks like the .bat file is returning an error.

[ilayn]

But nevermind, I already saw that it is building so that's good enough for me. I don't want to cause more work for you.

[h-vetinari]

I think this is not scipy related but something else is broken in the toolchain. Looks like the .bat file is returning an error.

Unfortunately, it's 100% guaranteed to be related to the changes that happened between 1.14 and your PR. If you want I can test SciPy main and see if it works as well, but you see a CI run further up where 1.14 passed CI today.

The fact that you see the failure in the bat script is just because that's the wrapper that executes the test suite. Unfortunately, it appears that the same kind of corruption also happens when we invoke pytest (probably during test collection, which already imports most of SciPy).

It's a frustrating kind of error to run into and to debug, but it's very real.

[ilayn]

Hmm then I have no idea what it might be and more perplexing that all passing in the SciPy CI. Frustrating indeed

[h-vetinari]

Good news (for your PR at least). SciPy main also fails in the same way...

[ilayn]

I commented there but here it is less noise I think, but the MSVC job of scipy is going through but that is using ifx. Is it a Flang problem possibility ?

[h-vetinari]

Is it a Flang problem possibility ?

Always a possibility, but not my first suspicion. Could also be a VS2019 vs. 2022 issue. We'll see what the bisection yields

[ilayn]

Do you mind adding the very last commit from scipy/scipy#21553 just to test if it will fail again? Then I would know how to recreate the failing test locally.

[h-vetinari]

CI retriggered, which should pick up the newest commits from that branch.

[ilayn]

Thank you, fingers crossed.

[ilayn]

Apologies Axel, I'm losing focus these days. I missed a few places that randomization was done in the old style. Could you retrigger this if you can please?

[h-vetinari]

I mentioned this in the SciPy PR already, but just to allow you to move at your own speed: you can retrigger CI here yourself with the following magic incantation (and because the sources point to your branch, that will pick up whatever new commits are in there):

@conda-forge-admin, please restart ci

[ilayn]

@conda-forge-admin, please restart ci

[ilayn]

@conda-forge-admin, please restart ci

[h-vetinari]

Congrats, bug squashed successfully! 😎

[rgommers]

Not sure if this mattered for previous builds or not, I suspect not since numpy is set centrally in conda-forge pinnings - but this change is more correct either way.

[h-vetinari]

It matters because we do the <2.{N+3} handling here, and this doesn't work with the global pinning (which now does major-only).

[rgommers]

The 1.14.1 package has this metadata:

 - numpy <2.4
 - numpy >=1.21,<3
 - numpy >=1.23.5

So it's already correct for previous builds (yay) - apparently numpy was defined in the conda-forge builds, or it would have said <2.3.

Anyway, this change is still helpful.

[rgommers]

Note that this was not true since the switch to Meson. We should probably switch back to sdist from PyPI, so we can avoid all this manual git submodule handling?

[h-vetinari]

It's a bit annoying to update the hashes, but I do prefer building from source. I really don't want to pick up anything pre-cythonized or otherwise precompiled. If that's not the case anymore, I'm happy to switch back to the upstream tarballs.

[rgommers]

There are no pre-generated sources except for the trivial capturing of the git hash that's being built from in version.py (that one is necessary because it's VCS info). And that one is actually broken in the current recipe, switching to a proper sdist will fix it:

>>> scipy.version.git_revision
'Unknown'

The GitHub-generated tarballs should never be used for Python packages if it can be avoided, because they're broken in multiple ways - this missing versioning info will happen for many Python packages.

[h-vetinari]

Could you please cherry pick Lucas' commit from #291?

[rgommers]

Could you please cherry pick Lucas' commit from #291?

Sure, will do after letting the current run complete (it's almost done).

[rgommers]

(it's almost done).

or not, these aarch64 are crazy slow, 3h 17min and going. we may want to start skipping tests marked as slow - those are tested on native aarch64 CI machines upstream after all.

[h-vetinari]

We're already only running the fast tests, but emulation is simply slow. It's not a big issue though, because we don't build so often. Just push, no need to wait for the runs to finish