[Upgrade to 1.6] ImportError: cannot import name unpack_url

[rbovard]

Following the doc upgrading from 1.5 to 1.6 [1], I have a Python error with make -f <makefile> project.yaml:

mkdir -p .build/
virtualenv --setuptools --no-site-packages .build/venv
Running virtualenv with interpreter /usr/bin/python2
New python executable in .build/venv/bin/python2
Not overwriting existing python script .build/venv/bin/python (you must use .build/venv/bin/python2)
Installing setuptools, pip...done.
.build/venv/bin/pip install \
        --index-url http://pypi.camptocamp.net/pypi \
        'pip>=7' 'setuptools>=12'
Traceback (most recent call last):
  File ".build/venv/bin/pip", line 9, in <module>
    load_entry_point('pip==7.1.0', 'console_scripts', 'pip')()
  File "/var/www/vhosts/nyon-geoportail/private/dev/.build/venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 558, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/var/www/vhosts/nyon-geoportail/private/dev/.build/venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2682, in load_entry_point
    return ep.load()
  File "/var/www/vhosts/nyon-geoportail/private/dev/.build/venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2355, in load
    return self.resolve()
  File "/var/www/vhosts/nyon-geoportail/private/dev/.build/venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2361, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/var/www/vhosts/nyon-geoportail/private/dev/.build/venv/lib/python2.7/site-packages/pip/__init__.py", line 76, in <module>
    from pip.commands import commands, get_summaries, get_similar_commands
  File "/var/www/vhosts/nyon-geoportail/private/dev/.build/venv/lib/python2.7/site-packages/pip/commands/__init__.py", line 6, in <module>
    from pip.commands.bundle import BundleCommand
  File "/var/www/vhosts/nyon-geoportail/private/dev/.build/venv/lib/python2.7/site-packages/pip/commands/bundle.py", line 6, in <module>
    from pip.commands.install import InstallCommand
  File "/var/www/vhosts/nyon-geoportail/private/dev/.build/venv/lib/python2.7/site-packages/pip/commands/install.py", line 5, in <module>
    from pip.req import InstallRequirement, RequirementSet, parse_requirements
  File "/var/www/vhosts/nyon-geoportail/private/dev/.build/venv/lib/python2.7/site-packages/pip/req/__init__.py", line 4, in <module>
    from .req_set import RequirementSet, Requirements
  File "/var/www/vhosts/nyon-geoportail/private/dev/.build/venv/lib/python2.7/site-packages/pip/req/req_set.py", line 12, in <module>
    from pip.download import (url_to_path, unpack_url)
ImportError: cannot import name unpack_url
CONST_Makefile:677: recipe for target '.build/venv.timestamp-noclean' failed
make: *** [.build/venv.timestamp-noclean] Error 1

Is something missing?

[1] http://docs.camptocamp.net/c2cgeoportal/1.6/integrator/upgrade_application.html#easy-upgrading-an-application-from-1-5-to-1-6

[ochriste]

can you access http://pypi.camptocamp.net/pypi/ from your server?
you can test with a "curl http://pypi.camptocamp.net/pypi/" from your server

[rbovard]

Yep

$ curl -I http://pypi.camptocamp.net/pypi/
HTTP/1.1 200 OK

[ochriste]

I presume you tried more than once ?

[rbovard]

Both of "curling" and getting the right version of the egg

[ochriste]

look in you .build/venv/lib/python2.7/site-packages/ folder and see which version of pip is installed there.
it is possible pip 1.5.x get installed by default and is causing problems.
if 1.5 is installed, remove it (delete both the pip and pip-1.5.6.dist-info folder) and install pip 7.1.0 using easy_install (.build/venv/bin/easy_install pip>=7)
then do a
.build/venv/bin/pip -V
and you should see
pip 7.1.0 from /home/ochriste/tmp/test/.build/venv/local/lib/python2.7/site-packages/pip-7.1.0-py2.7.egg (python 2.7)
(or equivalent)
then try to execute the commands that failed again

[rbovard]

In fact there is:

• pip
• pip-1.5.6.dist-info
• pip-7.1.0-py2.7.egg-info

I deleted those three and installed 7.1.0, it's ok:

pip 7.1.0 from /var/www/vhosts/nyon-geoportail/private/dev/.build/venv/local/lib/python2.7/site-packages/pip-7.1.0-py2.7.egg (python 2.7)

But now when trying the failing command, I get:

mkdir -p .build/
virtualenv --setuptools --no-site-packages .build/venv
Running virtualenv with interpreter /usr/bin/python2
New python executable in .build/venv/bin/python2
Not overwriting existing python script .build/venv/bin/python (you must use .build/venv/bin/python2)
Installing setuptools, pip...done.
.build/venv/bin/pip install \
        --index-url http://pypi.camptocamp.net/pypi \
        'pip>=7' 'setuptools>=12'
Requirement already satisfied (use --upgrade to upgrade): pip>=7 in ./.build/venv/lib/python2.7/site-packages/pip-7.1.0-py2.7.egg
Collecting setuptools>=12
  The repository located at pypi.camptocamp.net is not a trusted or secure host and is being ignored. If this repository is available via HTTPS it is recommended to use HTTPS instead, otherwise you may silence this warning and allow it anyways with '--trusted-host pypi.camptocamp.net'.
  Could not find a version that satisfies the requirement setuptools>=12 (from versions: )
No matching distribution found for setuptools>=12
CONST_Makefile:677: recipe for target '.build/venv.timestamp-noclean' failed
make: *** [.build/venv.timestamp-noclean] Error 1

[rbovard]

@ochriste Should I do something about setuptools?

[ochriste]

mmmm, I dont know for sure but you can try to install setuptools using pip:
.build/venv/bin/pip install setuptools>=12

theoretically you shouldnt have to do that as it should be able to find the package, but im short of idea

[ochriste]

ha sorry, I missed a message in the error:

the repository located at pypi.camptocamp.net is not a trusted or secure host and is being ignored.

thats the cause of the problem

[ochriste]

add --trusted-host pypi.camptocamp.net in the commands:

virtualenv --setuptools --no-site-packages --trusted-host pypi.camptocamp.net .build/venv

[rbovard]

Then new error when trying the failing command:

mkdir -p .build/
virtualenv --setuptools --no-site-packages .build/venv
Running virtualenv with interpreter /usr/bin/python2
New python executable in .build/venv/bin/python2
Not overwriting existing python script .build/venv/bin/python (you must use .build/venv/bin/python2)
Installing setuptools, pip...done.
.build/venv/bin/pip install \
        --index-url http://pypi.camptocamp.net/pypi \
        'pip>=7' 'setuptools>=12'
Requirement already satisfied (use --upgrade to upgrade): pip>=7 in ./.build/venv/lib/python2.7/site-packages
Downloading/unpacking setuptools>=12
  http://pypi.camptocamp.net/pypi/setuptools/ uses an insecure transport scheme (http). Consider using https if pypi.camptocamp.net has it available
  Downloading setuptools-18.1.zip (674kB): 674kB downloaded
  Running setup.py (path:/tmp/pip-build-LuTJ7S/setuptools/setup.py) egg_info for package setuptools

Installing collected packages: setuptools
  Found existing installation: setuptools 5.5.1
    Uninstalling setuptools:
      Successfully uninstalled setuptools
  Running setup.py install for setuptools

    Installing easy_install script to /var/www/vhosts/nyon-geoportail/private/dev/.build/venv/bin
    Installing easy_install-2.7 script to /var/www/vhosts/nyon-geoportail/private/dev/.build/venv/bin
Successfully installed setuptools
Cleaning up...
touch .build/venv.timestamp-noclean
.build/venv/bin/pip install --trusted-host pypi.camptocamp.net -r CONST_dev-requirements.txt 'tilecloud-chain>=1.0.0dev' JSTools 'c2c.cssmin>=0.7dev6'

Usage:
  pip install [options] <requirement specifier> ...
  pip install [options] -r <requirements file> ...
  pip install [options] [-e] <vcs project url> ...
  pip install [options] [-e] <local project path> ...
  pip install [options] <archive url/path> ...

no such option: --trusted-host
CONST_Makefile:673: recipe for target '.build/dev-requirements.timestamp' failed
make: *** [.build/dev-requirements.timestamp] Error 2

Is my server correctly configured or am I the first one to upgrade following the doc?

[rbovard]

Something related to pypa/pip#2822?

[ochriste]

install setuptools manually
.build/venv/bin/pip install setuptools>=12

[ochriste]

after that the last command indicated in the doc should work

[ochriste]

err

[ochriste]

if pip complain that it doesnt know --trusted-host, then it's not using the right version of pip.
try doing a "source .build/venv/bin/activate" before executing the ".build/venv/bin/pip install ...."

[ochriste]

I dont really understand though, since we use an absolute path to the pip executable :(

[rbovard]

Finally it worked!!

There is for sure some issues here, but with all installs and uninstalls I did, I can't tell you exactly what was wrong...

Thanks a lot @ochriste

[ochriste]

chances are you are probably right about it being related to pypa/pip#2822

[sbrunner]

Looks to work, closing