BUG: Python 3.13 development wheels not available

[lesteve]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

Look at https://anaconda.org/scientific-python-nightly-wheels/pandas/files, there is no cp313 wheels. There were some a few days ago but it seems like you are removing the old wheels on each upload (not 100% sure about this)? Last sucessful wheel build was July 26, see build log.

Issues seem to be related to Python 3.13 see build log (not Python 3.13 free-threaded in particular just to be clear)

Issue Description

In scikit-learn CI we have started relying on pandas development wheels for Python 3.13 free-threaded work. Probably this was a bit over-optimistic on our side, see scikit-learn/scikit-learn#29572 for more details.

In case of failures, we would be more than happy to rely on an older wheel. For example, if the wheel build has been failing for a week we would be more than happy to rely on the wheel that was uploaded a week ago in https://anaconda.org/scientific-python-nightly-wheels/pandas/files than not have any wheel available. I think this is what would happen for scikit-learn, numpy and scipy development wheels although I am not 100% sure about this. Of course, you may have very good reasons to do it like this ...

We could well revert scikit-learn/scikit-learn#29572 if you are telling us that things are a bit too early and test against pandas later when things are a bit more smooth.

Expected Behavior

There are some Python 3.13 wheels

Installed Versions

N/A

[mroeschke]

Yeah our wheel jobs depend on the unit tests 100% passing (this could be relaxed but there's a tradeoff of releasing a non-functioning nightly wheel).

The 3.13 wheels test against a numpy nighties which I think changed something that broke these unit tests, but no one has had time to look into the failure.

[lesteve]

OK thanks for the answer!

Actually looking a bit more, sorry for the noise, you do have the old wheels cp313-313t wheels see this (I only looked at the first page initially 😅).

Now I need to figure out why it seems like pip does not find the wheel somehow and tries to install from source in our CI ...

[mroeschke]

I'll try to address the Python 3.13 test failures by the end of the week so more recent wheels can be uploaded. It slipped my mind that these failures also affected our Python 3.13 nightlies

[lesteve]

OK thanks a lot, but just to be clear there is no strong pressure from the scikit-learn side on this, we are completely fine using an old pandas wheel for some time.

Originally I was confused and I thought that there was no cp313 wheel available. What was actually happening is that the sdist is more recent than the wheel, and so pip chose to install the sdist by default. Using pip install --only-binary :all: is the right way to convince pip to only use wheels and I have opened scikit-learn/scikit-learn#29598 to do just that.

[ogrisel]

Actually, since older builds seem to be garbage collected, there aren't any cp313t wheel for pandas dev anymore: scikit-learn/scikit-learn#29610 (comment)

[lysnikolaou]

It seems that most (all?) of the failures are related to a recent NumPy commit that appeared in the nightlies on July 24th. The NumPy commit that introduced the breakage is numpy/numpy@682cd04:

682cd040f0997ece83af51ffe0e3241242c7c1df is the first bad commit
commit 682cd040f0997ece83af51ffe0e3241242c7c1df (HEAD)
Author: Sebastian Berg <[email protected]>
Date:   Wed Jul 10 16:50:32 2024 +0200

    BUG,ENH: Fix generic scalar infinite recursion issues
    
    This reorganizes the avoidance of infinite recursion to happen
    in the generic scalar fallback, rather than attempting to do so
    (badly) in the scalarmath, where it only applies to longdouble
    to begin with.
    
    This was also needed for the follow up to remove special handling
    of python scalar subclasses.

 numpy/_core/src/multiarray/npy_static_data.c |   1 +
 numpy/_core/src/multiarray/npy_static_data.h |   1 +
 numpy/_core/src/multiarray/scalartypes.c.src | 311 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-------------
 numpy/_core/src/umath/scalarmath.c.src       |  16 ++---
 4 files changed, 276 insertions(+), 53 deletions(-)

[mroeschke]

The pandas CI is finally green wrt unit tests so our nightly wheel upload is finally working again

https://github.com/pandas-dev/pandas/actions/runs/10295469960
https://anaconda.org/scientific-python-nightly-wheels/pandas/files

so closing this out