Manual migration for PyPy3.7 on windows

[h-vetinari]

Like for pypy37, the migrator seems to have forgotten about numpy - it's nowhere to be found on the status page of the migration. Compare this comment from the pypy37 migration:

@isuruf: @conda-forge/bot, was there any code added to the bot to skip any packages in the migration yaml itself? Looks like the bot is skipping numpy for this migration

Therefore, let's do it manually to progress on the migration.

[conda-forge-linter]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

[h-vetinari]

@mattip, this seems to crash in core/tests/test_ufunc.py::TestUfunc::test_broadcast. Happy to have you push into this PR if you want to participate (would make you collaborator on my fork of this feedstock).

[h-vetinari]

Some good news - all that's separating us from a working PyPy build on windows is that one segfault in core/tests/test_ufunc.py::TestUfunc::test_broadcast, plus 4 test errors:

=========================== short test summary info ===========================
FAILED core/tests/test_numeric.py::TestClip::test_clip_property - hypothesis....
FAILED core/tests/test_scalarmath.py::TestConversion::test_int_from_infinite_longdouble
FAILED lib/tests/test_arraypad.py::TestStatistic::test_zero_stat_length_valid[median]
FAILED lib/tests/test_function_base.py::TestMedian::test_empty - AssertionErr...
= 4 failed, 15919 passed, 422 skipped, 1 deselected, 22 xfailed, 26 xpassed, 227 warnings in 1737.50s (0:28:57) =

and the hypothesis error can probably be ignored (or might just go away by itself):

Detailed log on the remaining failures

______________ TestConversion.test_int_from_infinite_longdouble _______________

self = <numpy.core.tests.test_scalarmath.TestConversion object at 0x000001f82b407948>

    def test_int_from_infinite_longdouble(self):
        # gh-627
        x = np.longdouble(np.inf)
        assert_raises(OverflowError, int, x)
        with suppress_warnings() as sup:
            sup.record(np.ComplexWarning)
            x = np.clongdouble(np.inf)
>           assert_raises(OverflowError, int, x)

..\_test_env\lib\site-packages\numpy\core\tests\test_scalarmath.py:428: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
..\_test_env\lib-python\3\unittest\case.py:756: in assertRaises
    return context.handle('assertRaises', args, kwargs)
..\_test_env\lib-python\3\unittest\case.py:178: in handle
    callable_obj(*args, **kwargs)
..\_test_env\lib-python\3\unittest\case.py:201: in __exit__
    self.obj_name))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <unittest.case._AssertRaisesContext object at 0x000001f82b404368>
standardMsg = 'OverflowError not raised by int'

    def _raiseFailure(self, standardMsg):
        msg = self.test_case._formatMessage(self.msg, standardMsg)
>       raise self.test_case.failureException(msg)
E       AssertionError: OverflowError not raised by int

..\_test_env\lib-python\3\unittest\case.py:135: AssertionError
______________ TestStatistic.test_zero_stat_length_valid[median] ______________

self = <numpy.lib.tests.test_arraypad.TestStatistic object at 0x000001f81a66fd70>
mode = 'median'

    @pytest.mark.filterwarnings("ignore:Mean of empty slice:RuntimeWarning")
    @pytest.mark.filterwarnings(
        "ignore:invalid value encountered in (true_divide|double_scalars):"
        "RuntimeWarning"
    )
    @pytest.mark.parametrize("mode", ["mean", "median"])
    def test_zero_stat_length_valid(self, mode):
        arr = np.pad([1., 2.], (1, 2), mode, stat_length=0)
        expected = np.array([np.nan, 1., 2., np.nan, np.nan])
>       assert_equal(arr, expected)
E       AssertionError: 
E       Arrays are not equal
E       
E       x and y nan location mismatch:
E        x: array([nan,  1.,  2.,  0.,  0.])
E        y: array([nan,  1.,  2., nan, nan])

..\_test_env\lib\site-packages\numpy\lib\tests\test_arraypad.py:484: AssertionError
____________________________ TestMedian.test_empty ____________________________

self = <numpy.lib.tests.test_function_base.TestMedian object at 0x000001f833e61210>

    def test_empty(self):
        # mean(empty array) emits two warnings: empty slice and divide by 0
        a = np.array([], dtype=float)
        with warnings.catch_warnings(record=True) as w:
            warnings.filterwarnings('always', '', RuntimeWarning)
            assert_equal(np.median(a), np.nan)
            assert_(w[0].category is RuntimeWarning)
            assert_equal(len(w), 2)
    
        # multiple dimensions
        a = np.array([], dtype=float, ndmin=3)
        # no axis
        with warnings.catch_warnings(record=True) as w:
            warnings.filterwarnings('always', '', RuntimeWarning)
>           assert_equal(np.median(a), np.nan)
E           AssertionError: 
E           Items are not equal:
E            ACTUAL: 0.0
E            DESIRED: nan

..\_test_env\lib\site-packages\numpy\lib\tests\test_function_base.py:3472: AssertionError

[mattip]

Thanks for the heads up. I will try to figure out what is broken.

[mattip]

It looks like the other windows runs are also not happy, the failures seem a bit random.

[h-vetinari]

Thanks for the heads up. I will try to figure out what is broken.

Cool, thanks a lot!

It looks like the other windows runs are also not happy, the failures seem a bit random.

Should be fixed by conda-forge/conda-forge-ci-setup-feedstock#156, hopefully

[h-vetinari]

Just recording another failure here (pypy+linux+ppc) that I haven't seen before:

=================================== FAILURES ===================================
_______ TestUfuncGenericLoops.test_unary_PyUFunc_O_O_method_full[floor] ________
self = <numpy.core.tests.test_ufunc.TestUfuncGenericLoops object at 0x00000cfb3ba11280>
ufunc = <ufunc 'floor'>
    @pytest.mark.parametrize("ufunc", UNARY_OBJECT_UFUNCS)
    def test_unary_PyUFunc_O_O_method_full(self, ufunc):
        """Compare the result of the object loop with non-object one"""
        val = np.float64(np.pi/4)
    
        class MyFloat(np.float64):
            def __getattr__(self, attr):
                try:
                    return super().__getattr__(attr)
                except AttributeError:
                    return lambda: getattr(np.core.umath, attr)(val)
    
        num_arr = np.array([val], dtype=np.float64)
        obj_arr = np.array([MyFloat(val)], dtype="O")
    
        with np.errstate(all="raise"):
            try:
                res_num = ufunc(num_arr)
            except Exception as exc:
                with assert_raises(type(exc)):
                    ufunc(obj_arr)
            else:
>               res_obj = ufunc(obj_arr)
E               FloatingPointError: underflow encountered in floor

[mattip]

Here is a minimal reproducer of the segfault:

scalar = np.timedelta64(123, "s")
dtype = np.int64
arr = np.ones((), dtype=dtype)
arr[()] = scalar

Now to dive in and figure out what is wrong, and why it succeeds on linux

[h-vetinari]

@mattip, this ran with the new pypy build, but unfortunately still segfaults in that test...

    pypy3.7:            7.3.5-hb504520_3         conda-forge
    python:             3.7.10-1_73_pypy         conda-forge
    python_abi:         3.7-2_pypy37_pp73        conda-forge

[h-vetinari]

So, another option would be to merge this with pypy --jit off in the test suite for now?

[mattip]

I think releasing a package that crashes in tests is risky.

Could we try to require openblas instead of mkl? Maybe there are some latent bugs in mkl on windows that pypy is exposing

[h-vetinari]

I think releasing a package that crashes in tests is risky.

There were no crashes, just the corner case failures of empty arrays we had talked about. Considering that pypy on windows (in conda-forge) has barely taken off yet, I think it would be likely be more useful to continue the migration and actually get to a point where a large(r) number of windows users can try it out - knowing that they're early adopters (most casual users don't know about pypy, from my experience). Which is not to say that the errors shouldn't be fixed, of course.

We can try using openblas in CI here, but MKL will still be the default on windows, so we wouldn't really be changing anything.

[isuruf]

I'm confused. #238 (comment) says there's a segfault.

[h-vetinari]

Nevermind, I should stop working this late. 🤦

My brain had already filed away the segfault as solved, but it isn't, of course. Apologies.

Regarding MKL, we can try if their most recent version fixes the issue as soon as conda-forge/blas-feedstock#74 is merged (we could pull this forward - blis & MKL are ready - otherwise we're waiting for some openblas issues around the 0.3.16 release to be resolved first).

[mattip]

I don't see anything in the mkl 2021.3 bugfixes that catches my eye. I ask about mkl because when I build locally on a avx512-enbled machine with openblas I do not get a segfault, when I build via the recipe I do get a segfault. Although now that I can reproduce the segfault, perhaps I can try to instrument the code to find exactly where the full_args struct is getting zeroed out.

[mattip]

Is there a way to discover which compiler built the MKL package?

[mattip]

@h-vetinari could you restart CI here to pick up the latest pypy fix?

[h-vetinari]

@h-vetinari could you restart CI here to pick up the latest pypy fix?

Sure, no problem (this weekend I just have very spotty availability). There's also a magic incantation that you can use on feedstock PRs without having to be maintainer, which is: "@ conda-forge-admin, please restart ci" [remove space after "@"].

[h-vetinari]

@mattip, good news, the new pypy build seems to have done the trick. 🥳
Even the empty-array corner-case failures are gone (and therefore seem to be due to some register shenanigans).

[mattip]

yay

[h-vetinari]

@conda-forge/numpy, this should now be ready (thanks to the tireless help of @mattip and others)!

As mentioned previously, I'm suggesting to add myself as a maintainer. Since the current migrator assumes assumes a 1.19 version of numpy also for pypy-on-windows, I'd offer to try backporting this. My plan would be to create a branch numpy119 (following the existing naming scheme for branches in this feedstock) at 14bfc88, which is the last commit before 1.20. I have a branch ready that would do the rest from there.

Assuming no opposition, I would proceed along those lines once this PR is merged.

[mattip]

According to oldest-supported-numpy the oldest numpy supported on pypy3.7 is numpy 1.20. I wouldn't spend time on trying to support 1.19 with pypy3.7 on any platform.

[h-vetinari]

According to oldest-supported-numpy the oldest numpy supported on pypy3.7 is numpy 1.20. I wouldn't spend time on trying to support 1.19 with pypy3.7 on any platform.

oldest-supported-numpy is not the reference for this. It comes from the global pinning feedstock (with the migrators layered on top), and you can see in the naming of the CI jobs that pypy3.7 (non-win) runs on numpy 1.19. In fact, since that field was not changed for the windows migrator, all open PRs waiting for numpy would need to be adapted (and the migrator updated). That's still the Plan B, but if a build on 1.19 ends up running through, that might even be the path of least effort.

[isuruf]

I think it's easier to just bump to 1.20 on pypy3.7 windows.

[h-vetinari]

I think it's easier to just bump to 1.20 on pypy3.7 windows.

I don't have a strong preference TBH (well, least amount of work, actually 😅), but whether 1.19 or 1.20, both need a backport. I have the branch for 1.19 ready and could try it out. If it doesn't work, redo for 1.20 - sounds reasonable?

[isuruf]

Ah, I didn't realize we were on 1.21. I thought master was 1.20.

[h-vetinari]

@isuruf, could you please push the button (unless we'd like to wait for more feedback)? 🙃

[rgommers]

🎉

As mentioned previously, I'm suggesting to add myself as a maintainer.

You already did in this PR, but for completeness: +1 from me. Thanks for all the work on this!