Running test suite on PyPy segfaults

[jakirkham]

It appears attempting to run the test suite on PyPy segfaults. See this CI build (also included the attached log for posterity). While this log is with Linux, this appears to happen on all OSes.

[lithomas1]

Pandas doesn't run well with PyPy. I think on the main repo we only basically test that it compiles too.

[jakirkham]

Gotcha. Should we instead track via an upstream issue then? Maybe this one ( pandas-dev/pandas#42509 )?

[h-vetinari]

Over in #162, I re-enabled tests on pypy (using the new builds from conda-forge/pypy3.6-feedstock#103), and the picture is pretty bad. Here's a snippet from the test suite (on linux) before it eventually hangs indefinitely:

........F...................F....................F...................... [ 28%]
.F...................F....................F.....................F....... [ 28%]
................F.....................F.................F............... [ 28%]
........F...................F.......................F...................F [ 28%]
......................F.........................F....................... [ 28%]
...F....................F........................F..................F... [ 28%]
...............F.F.F.......F..........F...........F..............F...... [ 28%]
..............F....................F..........FxxFFxFFFFFFFFFxFxxFxFxFxx [ 28%]
xx......xxFxFxFFxFxFxxFxxFFxFx..F.....................F........F..F..... [ 28%]
..F...FFFF..........F.........FF..F......F...........F...............Fss [ 28%]

Windows and other linux flavours also segfault, though on OSX we do get a result at least:

= 10371 failed, 160211 passed, 23994 skipped, 909 xfailed, 6 xpassed, 454 warnings, 21 errors in 6942.61s (1:55:42) =

So around 6% failures, most of which (at a glance) seem datetime related.

It's very unfortunate that we only had the most basic import tests at the time when #106 resp. #133 was merged. Hindsight is of course 20/20, but IMO pypy support should not have been merged here before we could get down to ≪50 non-essential test failures. The problem is we cannot take this decision back, as the pypy migration has rolled on assuming that pandas-on-pypy is functional, when it appears to be quite far from that.

I'm not sure when pypy 3.10 will materialize, nor when we will do that migration in conda-forge, but I don't think we should merge the migrator here unless we can get rid of the segfaults and reduce the test failures very substantially. Even if it means that a lot of packages that have pypy support now will not be available until we have those fixes in place.

Raising this now so we have some time to discuss/plan before that decision actually has to be made.

@conda-forge/pandas

CC @conda-forge/core

[WillAyd]

I think the datetime issues will trace back to what @mattip described in pandas-dev/pandas#50817 (comment)

[h-vetinari]

I think the datetime issues will trace back to what @mattip described in pandas-dev/pandas#50817 (comment)

The most recent run in #162 includes that fix already.

[mattip]

Truthfully, I am not convinced the ROI on PyPy migrations in conda-forge have proved worthy. Maybe I am mistaken, but I have not seen a lot of end users reporting back about their experience using the pypy conda-forge packages, either with kudos, thanks, or even with issues and complaints. I personally will not be pushing conda-forge to do a pypy3.10 migration, but would support the effort if there is a clear consensus that it is worthwhile.

As for the pandas segfaults: I will try to look into what is going on in #162.

[h-vetinari]

Truthfully, I am not convinced the ROI on PyPy migrations in conda-forge have proved worthy.

I genuinely think that even aside from download numbers & volume of user feedback, it has been a huge boost to systematically chew through the package ecosystem and improve (or even introduce) PyPy compatibility either in conda-forge or directly upstream.

That said, to me it appears to be a messaging problem first and foremost. The fundamental question is: how do people interested in PyPy find out that they can install it through conda-forge - currently it's only a sidenote on the installation page, without instructions.

Given the effort you've invested in ensuring a large part of the ecosystem can be installed out of the box with conda-forge (and how challenging it is in turn to get started with PyPy on your own), I think you should "advertise" the convenience of installation through conda-forge way harder, to attract people who aren't already comfortable with dealing with all the build problems themselves. But obviously I'm biased and how to position yourselves is a choice for the PyPy project to make...

[mattip]

I think you should "advertise" the convenience of installation through conda-forge way harder

I just now rewrote the installation page. I posted a blog post in November. The download page starts with mentioning conda. The release notes don't mention conda, so I guess we could blog post again once the conda builds are available. Other than that can you think of where else we could mention conda?

[h-vetinari]

Other than that can you think of where else we could mention conda?

I think the hands-on installation instructions (use these three lines if you have conda and you're off) is a great improvement. 👍

[mattip]

Pandas + PyPy no longer segfaults for me when running tests locally (on ubuntu x86_64). The tests are slow, and made much slower by coverage. There are a number of test failures, I am tracking them in a PyPy issue and on #162