Compatibility Issues with Python 3.7

[ghost]

Not a py-earth issue per se, but a more general compatibility issue between cython and Python 3.7. Has anyone found an acceptable workaround? Updating cython to latest release and rebuilding the .c files still produces the same issue as below for me.

ubuntu:~/py-earth$ python setup.py install
running install
running bdist_egg
running egg_info
writing sklearn_contrib_py_earth.egg-info/PKG-INFO
writing dependency_links to sklearn_contrib_py_earth.egg-info/dependency_links.txt
writing requirements to sklearn_contrib_py_earth.egg-info/requires.txt
writing top-level names to sklearn_contrib_py_earth.egg-info/top_level.txt
reading manifest file 'sklearn_contrib_py_earth.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'pyearth/test/pathological_data'
writing manifest file 'sklearn_contrib_py_earth.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
UPDATING build/lib.linux-x86_64-3.7/pyearth/_version.py
set build/lib.linux-x86_64-3.7/pyearth/_version.py to '0.1.0+0.gb209d19.dirty'
running build_ext
building 'pyearth._util' extension
gcc -pthread -B /home/ubuntu/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/ubuntu/anaconda3/lib/python3.7/site-packages/numpy/core/include -I/home/ubuntu/anaconda3/include/python3.7m -c pyearth/_util.c -o build/temp.linux-x86_64-3.7/pyearth/_util.o
In file included from /home/ubuntu/anaconda3/lib/python3.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1823:0,
from /home/ubuntu/anaconda3/lib/python3.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:18,
from /home/ubuntu/anaconda3/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h:4,
from pyearth/_util.c:495:
/home/ubuntu/anaconda3/lib/python3.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
#warning "Using deprecated NumPy API, disable it by "
^~~~~~~
pyearth/_util.c: In function ‘__Pyx__ExceptionSave’:
pyearth/_util.c:8071:21: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’?
*type = tstate->exc_type;
^~~~~~~~
curexc_type
pyearth/_util.c:8072:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’?
*value = tstate->exc_value;
^~~~~~~~~
curexc_value
pyearth/_util.c:8073:19: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
*tb = tstate->exc_traceback;
^~~~~~~~~~~~~
curexc_traceback
pyearth/_util.c: In function ‘__Pyx__ExceptionReset’:
pyearth/_util.c:8080:24: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’?
tmp_type = tstate->exc_type;
^~~~~~~~
curexc_type
pyearth/_util.c:8081:25: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’?
tmp_value = tstate->exc_value;
^~~~~~~~~
curexc_value
pyearth/_util.c:8082:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
tmp_tb = tstate->exc_traceback;
^~~~~~~~~~~~~
curexc_traceback
pyearth/_util.c:8083:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’?
tstate->exc_type = type;
^~~~~~~~
curexc_type
pyearth/_util.c:8084:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’?
tstate->exc_value = value;
^~~~~~~~~
curexc_value
pyearth/_util.c:8085:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
tstate->exc_traceback = tb;
^~~~~~~~~~~~~
curexc_traceback
pyearth/_util.c: In function ‘__Pyx__GetException’:
pyearth/_util.c:8140:24: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’?
tmp_type = tstate->exc_type;
^~~~~~~~
curexc_type
pyearth/_util.c:8141:25: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’?
tmp_value = tstate->exc_value;
^~~~~~~~~
curexc_value
pyearth/_util.c:8142:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
tmp_tb = tstate->exc_traceback;
^~~~~~~~~~~~~
curexc_traceback
pyearth/_util.c:8143:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’?
tstate->exc_type = local_type;
^~~~~~~~
curexc_type
pyearth/_util.c:8144:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’?
tstate->exc_value = local_value;
^~~~~~~~~
curexc_value
pyearth/_util.c:8145:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
tstate->exc_traceback = local_tb;
^~~~~~~~~~~~~
curexc_traceback

[jcrudy]

@kevdietz It seems to be an issue with cython, as you said. Specifically, the C code generated by cython is not compatible with python 3.7. See these cython issues:

cython/cython#1955
cython/cython#1978

I haven't tried, but it looks like using the latest master from the cython repo might fix the problem if updating to the latest release didn't.

Leaving this issue open for now and marking as a bug so others can more easily find it. If @kevdietz or anyone else confirms that this has been fixed on cython's end, please comment.

[ivan-bilan]

I have exactly the same issue, building Cython from the github repo doesn't solve the problem either.

[jcrudy]

@ivan-bilan @kevdietz When you build py-earth using setup.py, do you use the --cythonize argument? If not, could you try that? Without --cythonize, it just tries to compile the pre-built C files, which were generated with the old version of cython.

[jcrudy]

I just pushed a branch that should fix this issue. See the branch called issue191.

[ghost]

Many thanks @jcrudy . The branch fixed the issue for me.

[Hao883]

@ivan-bilan @kevdietz When you build py-earth using setup.py, do you use the --cythonize argument? If not, could you try that? Without --cythonize, it just tries to compile the pre-built C files, which were generated with the old version of cython.

Thank you. It works!

[ghost]

@wikihao py-earth would not build would not work with the --cythonize argument on Python 3.7. The branch that @jcrudy pushed on Dec 7 solved the issues for me. But you still need to use the --cythonize argument to compile the C files.

[jcrudy]

@kevdietz @wikihao If I remember correctly, all I really did in the branch was run

python setup.py build_ext --inplace --cythonize

with the latest master of cython. Possibly @wikihao is now using a cython version that's compatible with python 3.7, in which case it should be enough to use --cythonize on master. For those without a python 3.7 compatible cython, use the branch issue191 and do not use --cythonize. Hopefully I'll get time to do a new release eventually, which will eliminate the need for users to think about this problem. While this issue is still open, it means I probably haven't done that yet.

[ericjster]

FWIW, this syntax worked for me, installing with Pip. This tells pip to use branch "issue191". There is probably an equivalent syntax for requirements.txt.
pip install --editable git+https://github.com/scikit-learn-contrib/py-earth@issue191#egg=pyearth
then to use (just like examples):
from pyearth import Earth

[jcrudy]

@ericjster Thanks for posting that. The v0.2dev branch also has the fix for this issue, so

pip install --editable git+https://github.com/scikit-learn-contrib/[email protected]#egg=pyearth

should also work.

[jhsansom]

Is there any update when this development version or issue191 might be posted to PyPi? I'm trying to incorporate py-earth into another software package, but I cannot get setup.py to download this specific repository. I've tried different ways of using the link in install_requires and dependency_links, but I just cannot get it to work.

[jcrudy]

@jhsansom It will probably be some time unless someone other than me can volunteer to do it (if anyone reading this wants to, I will gladly advise). I guess you probably already tried the instructions here: https://mike.zwobble.org/2013/05/adding-git-or-hg-or-svn-dependencies-in-setup-py/. If not, please take a look.

If you get a chance, please post what you've tried so far and how it failed. Maybe I or someone else can help figure it out. At least it will save others the effort of trying the exact same things.