SystemError: NULL result without error in PyObject_Call

[ViktorStiskala]

There seems to be a problem when compiling greenlet==0.4.0 with GCC 4.8.
It randomly raises SystemError: NULL result without error in PyObject_Call or just segfaults. It works correctly when specifying CFLAGS="-O0".

More information together with stack trace can be found here: http://stackoverflow.com/questions/15924353/gevent-redis-py-systemerror-null-result-without-error-in-pyobject-call/15974280

[snaury]

Thank you for the bug report, could you please compile with gcc's -save-temps option and send me greenlet.s (assembly output), so I could see where's the problem?

[schmir]

this seems to be the same problem as in #29 (or at least the same fix works for me).

greenlet.s is to be found here:
https://gist.github.com/schmir/5384037
(amd64, gcc version 4.8.0 (Gentoo 4.8.0 p1.1, pie-0.5.5))

run-tests.py fails with the following on my machine:

Linking /home/ralf/greenlet/build/lib.linux-x86_64-2.7/greenlet.so to /home/ralf/greenlet/greenlet.so
python 2.7.3 (64 bit) using greenlet 0.4.0 from /home/ralf/greenlet/greenlet.so
test_arg_refs (tests.test_leaks.ArgRefcountTests) ... ERROR
test_kwarg_refs (tests.test_leaks.ArgRefcountTests) ... python: greenlet.c:379: slp_save_state: Assertion `owner->stack_saved == 0' failed.
zsh: abort      python run-tests.py

[snaury]

@schmir yes, it is very similar, gcc 4.8 chooses %ebx to keep the return value, which we save/restore ourselves and thus overwrite. And gcc is of course correct in doing that, our hacks are incorrect and flimsy. Making return value a "stack" variable may fix it now, but then gcc 4.9 or 4.10 would see thru that and break again. Ideally save/restore should be in assembly, so compilers can't mess with it, e.g. see how PyPy is doing it...

[ViktorStiskala]

Sorry for late response, hope this is what you've wanted.

https://gist.github.com/hareevs/5389807
https://gist.github.com/hareevs/5389816

gcc (GCC) 4.8.0, 3.8.6-1-ARCH #1 SMP PREEMPT Sat Apr 6 07:27:01 CEST 2013 x86_64 GNU/Linux

[ViktorStiskala]

Also, on my machine tests segfaults only sometimes: https://gist.github.com/hareevs/5389861

[schmir]

@snaury you're right, but at the moment I'm tempted to just compile with "-Os" since that will fix the issue now and I can't rewrite the code to use assembly language on my own (even for a single platform).

But then I didn't even benchmark the "-Os" change...

[ViktorStiskala]

Just wanted to confirm that CFLAGS="-O2 -fno-tree-dominator-opts referenced in #29 also works for me.

[schmir]

thanks hareevs!

I've pushed a fix to the wip-issue28-configure-compiler branch (9465db1).
As noted in #29 -Os doesn't fix the issue there, so I'm now checking if the compiler understand -fno-tree-dominator-opts...

@snaury: I think I'm going to merge #29 as is and try to fix amd64/x86 with the same change additionally. Does that make sense? Do you have any objections?

[snaury]

@schmir no, I don't have any objections, but I think the fix is only temporary. Eventunally compilers will add more optimizations and this is a game we can't win. There is a better way (https://github.com/pypy/pypy/blob/master/pypy/translator/c/src/stacklet/switch_x86_64_gcc.h), but I won't work on a similar approach until at least mid-May.

[schmir]

@snaury: ok great, you've got another year until the next major gcc release! Unless I start learning a bit of assembler I won't be of much help here...

[MeirKriheli]

Installed from master, but I still get segfaults from time to time, here's using the example from the docs:

In [4]: def test1():
   ...:         print 12
   ...:         gr2.switch()
   ...:         print 34
   ...:     

In [5]: def test2():
   ...:         print 56
   ...:         gr1.switch()
   ...:         print 78
   ...:     

In [6]: gr1 = greenlet(test1)

In [7]: gr2 = greenlet(test2)

In [8]: gr1.switch()
12
56
Segmentation fault (core dumped)

This is on Arch Linux 64 bit, with gcc 4.8.0.

[schmir]

The fixes had been committed to other branches. But now they are merged to master. So, please try again with the current master branch!

[MeirKriheli]

Hi,

Installed from master.

Looks good so far - segfaults no more.

Thanks