have_directory_for_build AssertionError when installing with --no-cache-dir (19.0.1)

[cjerdonek]

• pip version: 19.0.1

With the 19.0.1 release, some people are still reporting an AssertionError when installing with --no-cache-dir similar to the one in #6158. This issue is for the 19.0.1-specific issue as it is narrower in scope.

Here are details for a couple reports: #6158 (comment) and #6158 (comment)

The traceback looks like this:

Exception:
Traceback (most recent call last):
  File "/mnt/jenkins/workspace/venv/local/lib/python2.7/site-packages/pip/_internal/cli/base_command.py", line 176, in main
    status = self.run(options, args)
  File "/mnt/jenkins/workspace/venv/local/lib/python2.7/site-packages/pip/_internal/commands/install.py", line 346, in run
    session=session, autobuilding=True
  File "/mnt/jenkins/workspace/venv/local/lib/python2.7/site-packages/pip/_internal/wheel.py", line 886, in build
    assert have_directory_for_build
AssertionError

Some work-arounds (feel free to add to the list):

• Use pip version 18.1.
• Pass --no-use-pep517 in your invocation.
• See issue assertion when using --no-cache-dir in 19.0 #6158 for some other possibilities.

[pradyunsg]

ISTM this is only happening on Alpine Linux -- could someone confirm whether that is the case?

[vinayan3]

@pradyunsg I had problems as well. Please see my dup issue #6192

[dknepprath]

Same but Python 3.7

Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 176, in main
    status = self.run(options, args)
  File "/usr/local/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 346, in run
    session=session, autobuilding=True
  File "/usr/local/lib/python3.7/site-packages/pip/_internal/wheel.py", line 886, in build
    assert have_directory_for_build
AssertionError

Reverting to pip 18.1 worked for me.

[hrw]

ISTM this is only happening on Alpine Linux -- could someone confirm whether that is the case?

debian:stretch-backports official image:

INFO:kolla.common.utils.openstack-base:Traceback (most recent call last):
INFO:kolla.common.utils.openstack-base:  File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/pip/_internal/cli/base_command.py", line 176, in main
INFO:kolla.common.utils.openstack-base:    status = self.run(options, args)
INFO:kolla.common.utils.openstack-base:  File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/pip/_internal/commands/install.py", line 346, in run
INFO:kolla.common.utils.openstack-base:    session=session, autobuilding=True
INFO:kolla.common.utils.openstack-base:  File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/pip/_internal/wheel.py", line 886, in build
INFO:kolla.common.utils.openstack-base:    assert have_directory_for_build
INFO:kolla.common.utils.openstack-base:AssertionError

[gaborbernat]

@pfmoore I ran into this myself, basically just passing --no-cache breaks pip when discovering and installing a PEP-517 package. We need either to fallback to disable PEP-517 build, or instead use the TMPDIR env var.

Hitting this from jython that does not support pip cache mechanism inside virtualenv.

[eliasbrange]

Easily reproducable in docker:

# Same for 3.5.6, 3.6.8, etc.
FROM python:3.7.2-alpine

RUN pip3 install -U pip

RUN pip3 install --no-cache-dir pendulum

[gaborbernat]

The error is very cryptic here, I think pip should check before invoking wheel if the folder exists and raise a better readable error what folder is missing. At the moment this fails with an assertion error leaving the user clueless what folder is missing, making it harder to track down (e.g. incorrectly set pip cache dir, HOME env var, etc). It also does not state for which package build it failed (important if the install pulls in a lot of dependencies first).

[cosmith]

Using --no-use-pep517 didn't work for me, downgrading to 18.1 did (using FROM python:3.6-alpine)

[pradyunsg]

Using --no-use-pep517 didn't work for me, downgrading to 18.1 did (using FROM python:3.6-alpine)

Could you provide us a reproducing case for this issue?

[cjerdonek]

FYI, for the provided Dockerfile example, it is failing for:
<Link https://files.pythonhosted.org/packages/5b/57/71fc910edcd937b72aa0ef51c8f5734fbd8c011fa1480fce881433847ec8/pendulum-2.0.4.tar.gz#sha256=cf535d36c063575d4752af36df928882b2e0e31541b4482c97d63752785f9fcb (from https://pypi.org/simple/pendulum/) (requires-python:>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*)>
This means, for example, that the link.is_artifact check returns True:

pip/src/pip/_internal/wheel.py

Lines 876 to 879 in e5f4bbb

	elif autobuilding and req.link and not req.link.is_artifact:
	# VCS checkout. Build wheel just for this run.
	buildset.append((req, True))
	else:

If it were instead a VCS url, it looks like the assertion would succeed.

What is the purpose of the assertion, btw? Should the above case be working for a link like that?

[cjerdonek]

So I'm wondering if this logic might be incorrect / is the culprit:

pip/src/pip/_internal/wheel.py

Lines 884 to 886 in d9de37d

	if not _contains_egg_info(base):
	# E.g. local directory. Build wheel just for this run.
	ephem_cache = True

It seems like ephem_cache should also be set to True if that condition isn't true and no cache directory is available.

[cjerdonek]

FYI, I added a failing unit test for this issue here: #6219 (based on the Dockerfile example provided in this comment above)

Does anyone familiar with this part of the code know how the logic in compute_ephem_cache() should be changed to make the have_directory_for_build assertion succeed?

[pradyunsg]

@cjerdonek figured out a nice fix for this in #6219 and that's now merged. I'll make a 19.0.2 in the coming week, with that fix and more.

I'll go ahead and close this issue. I'll ping here when the release is made.

[pradyunsg]

pip 19.0.2 has been released with a fix for this.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.