flit_core uses pip-unsupported backend-path

[chrahunt]

Pip uses vendored dependencies. One step of the vendoring process involves invoking pip download --no-binary :all: --no-deps ... to get sdists in order to access the licenses. Running the script today has failed to download pep517 because its build system uses flit_core (here), which in version 2.0.2 uses the backend-path key in its pyproject.toml here which is not currently supported in the latest release pip (19.3.1).

Can we revert to takluyver/intreehooks in flit_core until a pip is released that implements backend-path?

Expectation:

Installing flit_core with --no-binary :all: succeeds

Outcome:

Installing flit_core with --no-binary :all: fails with a ModuleNotFoundError

t.sh

#!/bin/sh
cd "$(mktemp -d)"
python -m venv venv
./venv/bin/python -V
./venv/bin/python -m pip install --upgrade pip
./venv/bin/python -m pip install --no-binary :all: flit_core

Output

Python 3.8.0
Collecting pip
  Using cached https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 19.2.3
    Uninstalling pip-19.2.3:
      Successfully uninstalled pip-19.2.3
Successfully installed pip-19.3.1
Collecting flit_core
  Using cached https://files.pythonhosted.org/packages/89/cf/a76f37dfded167e97936b8d53308abe5a8d00b97d417a6a405e69167e685/flit_core-2.0.2.tar.gz
  Getting requirements to build wheel ... done
ERROR: Exception:
Traceback (most recent call last):
  File "/tmp/user/1000/tmp.Pc55uAkmB3/venv/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 153, in _main
    status = self.run(options, args)
  File "/tmp/user/1000/tmp.Pc55uAkmB3/venv/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 382, in run
    resolver.resolve(requirement_set)
  File "/tmp/user/1000/tmp.Pc55uAkmB3/venv/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 201, in resolve
    self._resolve_one(requirement_set, req)
  File "/tmp/user/1000/tmp.Pc55uAkmB3/venv/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 365, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/tmp/user/1000/tmp.Pc55uAkmB3/venv/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 312, in _get_abstract_dist_for
    abstract_dist = self.preparer.prepare_linked_requirement(
  File "/tmp/user/1000/tmp.Pc55uAkmB3/venv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 223, in prepare_linked_requirement
    abstract_dist = _get_prepared_distribution(
  File "/tmp/user/1000/tmp.Pc55uAkmB3/venv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 49, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(finder, build_isolation)
  File "/tmp/user/1000/tmp.Pc55uAkmB3/venv/lib/python3.8/site-packages/pip/_internal/distributions/source/legacy.py", line 37, in prepare_distribution_metadata
    self._setup_isolation(finder)
  File "/tmp/user/1000/tmp.Pc55uAkmB3/venv/lib/python3.8/site-packages/pip/_internal/distributions/source/legacy.py", line 90, in _setup_isolation
    reqs = backend.get_requires_for_build_wheel()
  File "/tmp/user/1000/tmp.Pc55uAkmB3/venv/lib/python3.8/site-packages/pip/_vendor/pep517/wrappers.py", line 151, in get_requires_for_build_wheel
    return self._call_hook('get_requires_for_build_wheel', {
  File "/tmp/user/1000/tmp.Pc55uAkmB3/venv/lib/python3.8/site-packages/pip/_vendor/pep517/wrappers.py", line 255, in _call_hook
    raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last):
  File "/tmp/user/1000/tmp.Pc55uAkmB3/venv/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 63, in _build_backend
    obj = import_module(mod_path)
  File "/home/chris/.pyenv/versions/3.8.0/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'flit_core'

Additional info:

• Related to Avoid generating metadata in pip download --no-deps ... pip#1884, which is requesting pip not invoke build backends when running with --no-deps

[takluyver]

It sounds like you've found a workaround for now (from your comment in pypa/pip#7354), and my PR on pip adds the missing bit that it needs to work with this. So I'm inclined to leave it as is, and push forwards with support for backend-path.

(When I made the release, I thought that pip would already support this - with the original API of pep517, pip would have picked up support automatically when it bundled the new version, and I had forgotten that the API changed)

[chrahunt]

In our case it was pretty straightforward to figure out, but I don't think it would be the same for most end users. Some of those queries would end up on the pip issue tracker and on projects using flit_core. Most people don't use --no-binary :all:, usually when repackaging or in enterprise environments, so it probably wouldn't be a big burden, but it wouldn't be nothing.

I don't know how acceptable my workaround (adding --only-binary flit_core to the command) would be in the general case.

With our default schedule, the PR (when merged) would end up in pip 20.0 (Jan 2020). @xavfernandez can comment on whether he is willing to make a 19.3.2 release to support this feature.

[xavfernandez]

With our default schedule, the PR (when merged) would end up in pip 20.0 (Jan 2020). @xavfernandez can comment on whether he is willing to make a 19.3.2 release to support this feature.

I'm lukewarm at the idea of a patch release adding a new functionality...
The next release does not seem that far (possibly ~ 6 weeks) and the use-case still relatively rare (I think).
If the need is urgent, I'd be more comfortable with a 19.4 release happening at the end of November and a pip 20.0 release in late January.

[LadyNamedLaura]

@takluyver this significantly breaks our build pipeline, we can no longer build any of our projects (without blocking out flit 2.0 from our local cache) because building entrypoints from source unconditionally pulls in the latest version of flit, even if flit 1.3 is already installed.
While I can fix this locally for us (by blocking 2.0.x from our caching proxy) but in general I don't really think waiting for a new pip release is a good solution for everybody.

[takluyver]

Sorry to hear that.

How about this: if my PR on pip to support the official mechanism lands, so I know I can go back to that soon, I'll do an interim release of flit which uses the workaround method until pip is released with that support. This is a feature defined in a standard, so it shouldn't be a controversial change to make it work.

[LadyNamedLaura]

@takluyver an interim release with a workaround would be lovely.
I get that ofc you want to use the advertised documented features of a standard. But pragmatically seen it's not ideal if that feature is (currently) broken in one of the bigger implementations of that standard.
I wish you the best of luck with your PR. more standardization and better adherence to specs is for the better of the whole community.

[takluyver]

My pip PR was merged, so as promised, flit 2.1 is out using intreehooks. I'll go back to backend-path once a released version of pip supports that.

Flit 2.1 also compresses the files in wheels, which apparently I had had turned off for some time. Kudos to @dholth for spotting that.

[chrahunt]

@takluyver thanks again for your help. pip 20.0, released 2020/01/21, includes the fix for backend-path.

[takluyver]

Thanks @chrahunt - I've opened issue #316 to go back to backend-path.