libcurl link-time ssl backend (nss) is different from compile-time

[piuma]

Steps to reproduce:

$ python3 -m venv .

$ . ./bin/activate

$ pip install wptools
Collecting wptools
  Using cached wptools-0.3-py2.py3-none-any.whl
Collecting html2text (from wptools)
  Using cached html2text-2016.9.19.tar.gz
Collecting pycurl (from wptools)
  Using cached pycurl-7.43.0.tar.gz
Collecting certifi (from wptools)
  Using cached certifi-2017.7.27.1-py2.py3-none-any.whl
Collecting lxml (from wptools)
  Using cached lxml-3.8.0-cp35-cp35m-manylinux1_x86_64.whl
Installing collected packages: html2text, pycurl, certifi, lxml, wptools
  Running setup.py install for html2text ... done
  Running setup.py install for pycurl ... done
Successfully installed certifi-2017.7.27.1 html2text-2016.9.19 lxml-3.8.0 pycurl-7.43.0 wptools-0.3
You are using pip version 8.1.2, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

$ python
Python 3.5.3 (default, May 10 2017, 15:05:55) 
[GCC 6.3.1 20161221 (Red Hat 6.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import wptools
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mazzetta/prog/howoldis/lib64/python3.5/site-packages/wptools/__init__.py", line 23, in <module>
    from . import fetch
  File "/home/mazzetta/prog/howoldis/lib64/python3.5/site-packages/wptools/fetch.py", line 20, in <module>
    import pycurl
ImportError: pycurl: libcurl link-time ssl backend (nss) is different from compile-time ssl backend (none/other)

[piuma]

In my Linux Fedora distibution works after reinstall libcurl by running this command
pip install --no-cache-dir --compile --ignore-installed --install-option="--with-nss" pycurl

[siznax]

Thanks for trying wptools, @piuma!

Currently, we don't specify an SSL backend in the build process, and we build our distribution on Mac OS X:

>>> pycurl.version_info()[5]
'SecureTransport'

The pycurl install doc recommends OpenSSL and points out that you can specify which SSL backend on setup:

$ python setup.py --with-[openssl|gnutls|nss] install

and, apparently, you can be specific in the requirements.txt file:

pycurl>=7.43.0 --global-option="--with-openssl"

However, it seems best not to be specific. Your solution seems good, and others can try:

(env)$ pip uninstall pycurl
(env)$ pip install --upgrade pip
(env)$ export PYCURL_SSL_LIBRARY=[openssl|gnutls|nss|ssl]
(env)$ pip install pycurl

If anyone knows of a way we could adjust our build process to make the distribution more robust, please let us know here.

[siznax]

I ran into this issue after upgrading to Mac OS 10.13 High Sierra, and this worked for me:

Upgrade openssl

$ brew upgrade openssl
...
==> Pouring openssl-1.0.2m.high_sierra.bottle.tar.gz
==> Caveats
A CA file has been bootstrapped using certificates from the SystemRoots
keychain. To add additional certificates (e.g. the certificates added in
the System keychain), place .pem files in
  /usr/local/etc/openssl/certs

and run
  /usr/local/opt/openssl/bin/c_rehash

This formula is keg-only, which means it was not symlinked into /usr/local,
because Apple has deprecated use of OpenSSL in favor of its own TLS and crypto libraries.

If you need to have this software first in your PATH run:
  echo 'export PATH="/usr/local/opt/openssl/bin:$PATH"' >> ~/.bash_profile

For compilers to find this software you may need to set:
    LDFLAGS:  -L/usr/local/opt/openssl/lib
    CPPFLAGS: -I/usr/local/opt/openssl/include
For pkg-config to find this software you may need to set:
    PKG_CONFIG_PATH: /usr/local/opt/openssl/lib/pkgconfig

==> Summary
🍺  /usr/local/Cellar/openssl/1.0.2m: 1,792 files, 12.3MB

Export flags and rebuild

[wptools]$ export LDFLAGS="-L/usr/local/opt/openssl/lib"
[wptools]$ export CPPFLAGS="-I/usr/local/opt/openssl/include"
[wptools]$ pip install --no-cache-dir --compile --ignore-installed --install-option="--with-openssl" pycurl

[padodev]

@siznax Have the same problem after upgrading to High Sierra. But when I try to use your pip install command, it doesn't appear to be using CPPFLAGS

% ls /usr/local/opt/openssl/include/openssl/crypto.h
/usr/local/opt/openssl/include/openssl/crypto.h

% env | grep CPPFLAGS
CPPFLAGS=-I/usr/local/opt/openssl/include

% sudo -H pip install --no-cache-dir --compile --ignore-installed --install-option="--with-openssl" pycurl
/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pip/commands/install.py:194: UserWarning: Disabling all use of wheels due to the use of --build-options / --global-options / --install-options.
cmdoptions.check_install_build_global(options)
Collecting pycurl
Downloading pycurl-7.43.0.tar.gz (182kB)
100% |████████████████████████████████| 184kB 3.2MB/s
Installing collected packages: pycurl
Running setup.py install for pycurl ... error
Complete output from command /opt/local/bin/python3 -u -c "import setuptools, tokenize;file='/private/tmp/pip-build-k6eijxf6/pycurl/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-w2tj80un-record/install-record.txt --single-version-externally-managed --compile --with-openssl:
Using curl-config (libcurl 7.54.0)
running install
running build
running build_py
creating build
creating build/lib.macosx-10.13-x86_64-3.5
creating build/lib.macosx-10.13-x86_64-3.5/curl
copying python/curl/init.py -> build/lib.macosx-10.13-x86_64-3.5/curl
running build_ext
building 'pycurl' extension
creating build/temp.macosx-10.13-x86_64-3.5
creating build/temp.macosx-10.13-x86_64-3.5/src
/usr/bin/clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -pipe -Os -DPYCURL_VERSION="7.43.0" -DHAVE_CURL_OPENSSL=1 -DHAVE_CURL_SSL=1 -I/opt/local/Library/Frameworks/Python.framework/Versions/3.5/include/python3.5m -c src/docstrings.c -o build/temp.macosx-10.13-x86_64-3.5/src/docstrings.o
In file included from src/docstrings.c:4:
src/pycurl.h:170:13: fatal error: 'openssl/crypto.h' file not found
# include <openssl/crypto.h>
^~~~~~~~~~~~~~~~~~
1 error generated.
error: command '/usr/bin/clang' failed with exit status 1

[siznax]

@padodev, weird. I guess I would try it in a virtualenv, and also export LDFLAGS? And, check that your pip is up-to-date.

[padodev]

Yes, LDFLAGS is exported

% env | grep LDFLAGS
LDFLAGS=-L/usr/local/opt/openssl/lib

[ilovefood2]

I have same issue here,im on high Sierra
running
sudo pip3 install --no-cache-dir --install-option="--with-openssl" pycurl
gives error as following

`Running setup.py install for pycurl ... error
Complete output from command /Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 -u -c "import setuptools, tokenize;file='/private/tmp/pip-build-klzshss8/pycurl/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-0a5254vp-record/install-record.txt --single-version-externally-managed --compile --with-openssl:
Using curl-config (libcurl 7.54.0)
running install
running build
running build_py
creating build
creating build/lib.macosx-10.6-intel-3.6
creating build/lib.macosx-10.6-intel-3.6/curl
copying python/curl/init.py -> build/lib.macosx-10.6-intel-3.6/curl
running build_ext
building 'pycurl' extension
creating build/temp.macosx-10.6-intel-3.6
creating build/temp.macosx-10.6-intel-3.6/src
/usr/bin/clang -fno-strict-aliasing -Wsign-compare -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -DPYCURL_VERSION="7.43.0.1" -DHAVE_CURL_OPENSSL=1 -DHAVE_CURL_SSL=1 -I/Library/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c src/docstrings.c -o build/temp.macosx-10.6-intel-3.6/src/docstrings.o
In file included from src/docstrings.c:4:
src/pycurl.h:164:13: fatal error: 'openssl/ssl.h' file not found
# include <openssl/ssl.h>
^~~~~~~~~~~~~~~
1 error generated.
error: command '/usr/bin/clang' failed with exit status 1

----------------------------------------

Command "/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 -u -c "import setuptools, tokenize;file='/private/tmp/pip-build-klzshss8/pycurl/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-0a5254vp-record/install-record.txt --single-version-externally-managed --compile --with-openssl" failed with error code 1 in /private/tmp/pip-build-klzshss8/pycurl/
`

[nguillermin]

For @ikelca and all those who stumble here, try this move from the pycurl github tracker pycurl/pycurl#530

Thanks, installing with PYCURL_SSL_LIBRARY=openssl works, but only if I point pip also to an include directory for openssl header files, because Apple doesn't seem to ship the one for the version of LibreSSL they're using with macOS High Sierra. So I had to install pycurl with

pip install pycurl --global-option="build_ext" --global-option="-I/usr/local/opt/openssl/include"

(I'm not completely sure, but I suppose that include directory was installed by Homebrew.)

[mahesh-wabale]

pip install --no-cache-dir --compile --ignore-installed --install-option="--with-nss" pycurl

Works for me for centos6

[siznax]

If nothing above works for you, see:

• https://cscheng.info/2018/01/26/installing-pycurl-on-macos-high-sierra.html
• ImportError: pycurl: libcurl link-time ssl backend (openssl) is different from compile-time ssl backend (none/other) transloadit/python-sdk#4

[NTUEECS2014]

Thanks for trying wptools, @piuma!

Currently, we don't specify an SSL backend in the build process, and we build our distribution on Mac OS X:

>>> pycurl.version_info()[5]
'SecureTransport'

The pycurl install doc recommends OpenSSL and points out that you can specify which SSL backend on setup:

$ python setup.py --with-[openssl|gnutls|nss] install

and, apparently, you can be specific in the requirements.txt file:

pycurl>=7.43.0 --global-option="--with-openssl"

However, it seems best not to be specific. Your solution seems good, and others can try:

(env)$ pip uninstall pycurl
(env)$ pip install --upgrade pip
(env)$ export PYCURL_SSL_LIBRARY=[openssl|gnutls|nss|ssl]
(env)$ pip install pycurl

If anyone knows of a way we could adjust our build process to make the distribution more robust, please let us know here.

I solved the problem by using --global-option="--with-openssl". Thanks a lot.