Support python3.5

[kracekumar]

(http_framework_probe)➜  http_framework_probe  python3.5
Python 3.5.0 (default, Sep 23 2015, 04:42:00)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

(http_framework_probe)➜  http_framework_probe  pip freeze | grep greenlet
greenlet==0.4.9
(http_framework_probe)➜  http_framework_probe  ls /Users/krace/.virtualenvs/http_framework_probe/include/python3.5m/ | grep greenlet
(http_framework_probe)➜  http_framework_probe  CFLAGS="-I/Users/krace/.virtualenvs/http_framework_probe/include/python3.5m" UWSGI_PROFILE="asyncio" pip3 install uwsgi
Collecting uwsgi
  Using cached uwsgi-2.0.11.2.tar.gz
Building wheels for collected packages: uwsgi
  Building wheel for uwsgi failed: [Errno 13] Permission denied: '/Users/krace/Library/Caches/pip/wheels/a7'
Failed to build uwsgi
Installing collected packages: uwsgi
  Running setup.py install for uwsgi
    Complete output from command /Users/krace/.virtualenvs/http_framework_probe/bin/python3.5 -c "import setuptools, tokenize;__file__='/private/var/folders/xk/8lhz2_j91_3758cc65stj5z40000gn/T/pip-build-_kuhkrd1/uwsgi/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/xk/8lhz2_j91_3758cc65stj5z40000gn/T/pip-vd46d590-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/krace/.virtualenvs/http_framework_probe/bin/../include/site/python3.5/uwsgi:
    running install
    plugins/greenlet/greenlet.c:2:10: fatal error: 'greenlet/greenlet.h' file not found
    #include <greenlet/greenlet.h>
             ^
    1 error generated.

    ----------------------------------------
Command "/Users/krace/.virtualenvs/http_framework_probe/bin/python3.5 -c "import setuptools, tokenize;__file__='/private/var/folders/xk/8lhz2_j91_3758cc65stj5z40000gn/T/pip-build-_kuhkrd1/uwsgi/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/xk/8lhz2_j91_3758cc65stj5z40000gn/T/pip-vd46d590-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/krace/.virtualenvs/http_framework_probe/bin/../include/site/python3.5/uwsgi" failed with error code 1 in /private/var/folders/xk/8lhz2_j91_3758cc65stj5z40000gn/T/pip-build-_kuhkrd1/uwsgi

pip install greenlet succeeds but greenlet.h isn't placed in include directory as a result uwsgi installation fails.

[snaury]

I will look into it, but I'm not sure what could be wrong on greenlet's part, since greenlet.h is specified in headers.

[snaury]

I think the problem is either with Python 3.5 on OSX, or pip/setuptools. When I brew install python3, then pip3 install greenlet installs header as /usr/local/include/python3.5m/greenlet/greenlet.h. However ``/usr/local/include/python3.5mis not among includes when building other extension modules. Worse, when usingpyvenv myenv` and `myenv/bin/pip install greenlet`, then header is installed as `myenv/include/site/python3.5/greenlet/greenlet.h`, and again that directory is not among includes when building extension modules (only `myenv/include` is). Something is very broken and it's not greenlet's fault. I wonder if it's any different on Linux... wouldn't be the first time Python is broken on OSX.

[snaury]

Ok, so on Linux there's no problem when installing "globally" (I built 3.5 with $home/python-3.5.0 as $prefix, and since other Python includes are in $prefix/include/python3.5m it naturally works). This case might be failing on OSX due to Python headers going into a separate framework, but it might also be due to some homebrew customizations, I'm not sure.

However there's the same problem with pyvenv on Linux, headers are installed in a weird subdirectory that is not included during extension builds. That seems like a problem with pyvenv.

Both of these problems are unrelated to greenlet and should be reported to http://bugs.python.org. Would you please do that, and comment with issue links here for reference? Thanks.

[kracekumar]

Thanks @snaury. Yes, this happens with venv. I created venv using mkvirtualenv.

➜  ~  mkvirtualenv --version
13.1.2

I will report to https://bugs.python.org and post relevant links.

[kracekumar]

One more thing I noticed wheel fails to build.

(venv_bug)$/tmp  pip install greenlet
Collecting greenlet
  Using cached greenlet-0.4.9.tar.gz
Building wheels for collected packages: greenlet
  Running setup.py bdist_wheel for greenlet
  Complete output from command /private/tmp/venv_bug/bin/python3.5 -c "import setuptools;__file__='/private/var/folders/xk/8lhz2_j91_3758cc65stj5z40000gn/T/pip-build-sxtdlufe/greenlet/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /var/folders/xk/8lhz2_j91_3758cc65stj5z40000gn/T/tmpf6g8rwt_pip-wheel-:
  running bdist_wheel
  running build
  running build_ext
  building 'greenlet' extension
  creating build
  creating build/temp.macosx-10.10-x86_64-3.5
  clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/include/python3.5m -c greenlet.c -o build/temp.macosx-10.10-x86_64-3.5/greenlet.o
  creating build/lib.macosx-10.10-x86_64-3.5
  clang -bundle -undefined dynamic_lookup build/temp.macosx-10.10-x86_64-3.5/greenlet.o -o build/lib.macosx-10.10-x86_64-3.5/greenlet.cpython-35m-darwin.so
  installing to build/bdist.macosx-10.10-x86_64/wheel
  running install
  running install_lib
  creating build/bdist.macosx-10.10-x86_64
  creating build/bdist.macosx-10.10-x86_64/wheel
  copying build/lib.macosx-10.10-x86_64-3.5/greenlet.cpython-35m-darwin.so -> build/bdist.macosx-10.10-x86_64/wheel
  running install_headers
  creating build/bdist.macosx-10.10-x86_64/wheel/greenlet-0.4.9.data
  creating build/bdist.macosx-10.10-x86_64/wheel/greenlet-0.4.9.data/headers
  copying greenlet.h -> build/bdist.macosx-10.10-x86_64/wheel/greenlet-0.4.9.data/headers
  running install_egg_info
  running egg_info
  creating greenlet.egg-info
  writing top-level names to greenlet.egg-info/top_level.txt
  writing greenlet.egg-info/PKG-INFO
  writing dependency_links to greenlet.egg-info/dependency_links.txt
  writing manifest file 'greenlet.egg-info/SOURCES.txt'
  warning: manifest_maker: standard file '-c' not found

  reading manifest file 'greenlet.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  writing manifest file 'greenlet.egg-info/SOURCES.txt'
  Copying greenlet.egg-info to build/bdist.macosx-10.10-x86_64/wheel/greenlet-0.4.9-py3.5.egg-info
  running install_scripts
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/private/var/folders/xk/8lhz2_j91_3758cc65stj5z40000gn/T/pip-build-sxtdlufe/greenlet/setup.py", line 101, in <module>
      **setuptools_args)
    File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py", line 955, in run_commands
      self.run_command(cmd)
    File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py", line 974, in run_command
      cmd_obj.run()
    File "/private/tmp/venv_bug/lib/python3.5/site-packages/wheel/bdist_wheel.py", line 213, in run
      archive_basename = self.get_archive_basename()
    File "/private/tmp/venv_bug/lib/python3.5/site-packages/wheel/bdist_wheel.py", line 161, in get_archive_basename
      impl_tag, abi_tag, plat_tag = self.get_tag()
    File "/private/tmp/venv_bug/lib/python3.5/site-packages/wheel/bdist_wheel.py", line 155, in get_tag
      assert tag == supported_tags[0]
  AssertionError

  ----------------------------------------
  Failed building wheel for greenlet
Failed to build greenlet
Installing collected packages: greenlet
  Running setup.py install for greenlet
Successfully installed greenlet-0.4.9

Python bug url: https://bugs.python.org/issue25531

[snaury]

Not sure what that bdist_wheel assertion is about (I can't reproduce it, everything builds cleanly with pip install greenlet -v, but I don't use bottled builds and build everything from source, and I'm using OSX 10.11.1 with latest Xcode, so maybe that's why). Try removing ~/Library/Caches/pip just in case.

[kracekumar]

Clearing cache directory or passing --no-cache-dir throws same error. I am on older version of OS X. I will wait for reply in CPython bug tracker and update.

[kracekumar]

I am able to build uwsgi, you can find more details here https://github.com/unbit/uwsgi/issues/1100.

Here is a quick way to check.

➜  /tmp  python3.5 -m venv py3.5 uwsgi
➜  /tmp  . py3.5/bin/activate
(py3.5) ➜  /tmp  pip install greenlet
Collecting greenlet
Installing collected packages: greenlet
Successfully installed greenlet-0.4.9
(py3.5) ➜  /tmp  CC=clang CFLAGS="-I/tmp/py3.5/include/site/python3.5" UWSGI_PROFILE="asyncio" python -m pip --verbose --verbose --verbose install uwsgi
....
*** uWSGI is ready, launch it with /private/tmp/py3.5/bin/uwsgi ***
(py3.5) ➜  /tmp  /private/tmp/py3.5/bin/uwsgi --asyncio 10
*** Starting uWSGI 2.0.11.2 (64bit) on [Thu Nov  5 01:47:08 2015] ***
compiled with version: 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72) on 05 November 2015 01:46:20
os: Darwin-14.5.0 Darwin Kernel Version 14.5.0: Wed Jul 29 02:26:53 PDT 2015; root:xnu-2782.40.9~1/RELEASE_X86_64
nodename: Kracekumars-MacBook-Pro.local
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 4
current working directory: /private/tmp
detected binary path: /private/tmp/py3.5/bin/uwsgi
*** WARNING: you are running uWSGI without its master process manager ***
your processes number limit is 709
your memory page size is 4096 bytes
detected max file descriptor number: 256
- async cores set to 10 - fd table size: 256
lock engine: OSX spinlocks
thunder lock: disabled (you can enable it with --thunder-lock)
Python version: 3.5.0 (default, Sep 23 2015, 04:42:00)  [GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)]
Python main interpreter initialized at 0x7ff139409770
python threads support enabled
your mercy for graceful operations on workers is 60 seconds
mapped 165424 bytes (161 KB) for 10 cores
*** Operational MODE: async ***
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (and the only) (pid: 2334, cores: 10)
*** running asyncio loop engine [addr:0x10bcf3f50] ***
*** DANGER *** asyncio mode without coroutine/greenthread engine loaded !!!
^C%