Wheel 0.34 broke installing in docker-alpine #332

andronov04 · 2020-01-27T12:40:23Z

When i build docker image I get an error below

Use:

python:3.7.6-alpine3.10
Docker version 19.03.5, build 633a0ea

      Collecting setuptools>=34.4
        Using cached https://files.pythonhosted.org/packages/42/3e/2464120172859e5d103e5500315fb5555b1e908c0dacc73d80d35a9480ca/setuptools-45.1.0.zip
      Collecting wheel
        Using cached https://files.pythonhosted.org/packages/15/39/728ccb81172c73f14084948b2d4561d5e51649dcf4b21d1f207b7363b1d7/wheel-0.34.0.tar.gz
      ERROR: Exception:
      Traceback (most recent call last):
        File "/usr/local/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 188, in main
          status = self.run(options, args)
        File "/usr/local/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 345, in run
          resolver.resolve(requirement_set)
        File "/usr/local/lib/python3.6/site-packages/pip/_internal/legacy_resolve.py", line 196, in resolve
          self._resolve_one(requirement_set, req)
        File "/usr/local/lib/python3.6/site-packages/pip/_internal/legacy_resolve.py", line 359, in _resolve_one
          abstract_dist = self._get_abstract_dist_for(req_to_install)
        File "/usr/local/lib/python3.6/site-packages/pip/_internal/legacy_resolve.py", line 307, in _get_abstract_dist_for
          self.require_hashes
        File "/usr/local/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 213, in prepare_linked_requirement
          with self.req_tracker.track(req):
        File "/usr/local/lib/python3.6/contextlib.py", line 81, in __enter__
          return next(self.gen)
        File "/usr/local/lib/python3.6/site-packages/pip/_internal/req/req_tracker.py", line 94, in track
          self.add(req)
        File "/usr/local/lib/python3.6/site-packages/pip/_internal/req/req_tracker.py", line 63, in add
          % (link, fp.read()))
      LookupError: https://files.pythonhosted.org/packages/15/39/728ccb81172c73f14084948b2d4561d5e51649dcf4b21d1f207b7363b1d7/wheel-0.34.0.tar.gz#sha256=a69231973dccde8558f80f2ef058335a1d0bc1b55b2a5a2dfb5097e151e3c498 (from https://pypi.org/simple/wheel/) (requires-python:!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7) is already being built: wheel from https://files.pythonhosted.org/packages/15/39/728ccb81172c73f14084948b2d4561d5e51649dcf4b21d1f207b7363b1d7/wheel-0.34.0.tar.gz#sha256=a69231973dccde8558f80f2ef058335a1d0bc1b55b2a5a2dfb5097e151e3c498

The text was updated successfully, but these errors were encountered:

LeonJansen · 2020-01-27T12:46:32Z

Same error here!

agronholm · 2020-01-27T14:28:39Z

This error comes from the internals of pip. Do you mind digging deeper to figure out what the LookupError is about? I can repro this outside of Docker with pip install --user wheel==0.34.0 --no-binary=wheel.

tomasmor42 · 2020-01-27T14:33:32Z

I can try to help. Could you elaborate on what exactly do you mean by digging deeper?

jamadden · 2020-01-27T14:34:41Z

Here's another example, taken from pip install --no-binary :all: ./wheel-0.34.0.tar.gz. It looks like a dependency resolution loop. The setup_requires that wheel has on setuptools_scm[toml] is leading to trying to install wheel, because setuptools_scm has an setup_requires on wheel. (This is all in pyroject.toml files.)

$ pip install --no-binary :all: ./wheel-0.34.0.tar.gz
Processing ./wheel-0.34.0.tar.gz
  Installing build dependencies ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/bin/python3.8 /Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /private/var/folders/y5/x7pvzk651c3dqkllbxd1jd280000gn/T/pip-build-env-eewzron7/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple --pre -- 'setuptools >= 42' 'setuptools_scm[toml]>=3.4'
       cwd: None
  Complete output (51 lines):
  Collecting setuptools>=42
    Downloading setuptools-45.1.0.zip (859 kB)
  Collecting setuptools_scm[toml]>=3.4
    Downloading setuptools_scm-3.4.3.tar.gz (45 kB)
    Installing build dependencies: started
    Installing build dependencies: finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: /Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/bin/python3.8 /Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /private/var/folders/y5/x7pvzk651c3dqkllbxd1jd280000gn/T/pip-build-env-v8z5aszd/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple --pre -- 'setuptools>=34.4' wheel
         cwd: None
    Complete output (39 lines):
    Collecting setuptools>=34.4
      Using cached setuptools-45.1.0.zip (859 kB)
    Collecting wheel
      Downloading wheel-0.34.0.tar.gz (60 kB)
      Installing build dependencies: started
      Installing build dependencies: finished with status 'error'
      ERROR: Command errored out with exit status 2:
       command: /Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/bin/python3.8 /Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /private/var/folders/y5/x7pvzk651c3dqkllbxd1jd280000gn/T/pip-build-env-6fxxxyn6/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple --pre -- 'setuptools >= 42' 'setuptools_scm[toml]>=3.4'
           cwd: None
      Complete output (27 lines):
      Collecting setuptools>=42
        Using cached setuptools-45.1.0.zip (859 kB)
      Collecting setuptools_scm[toml]>=3.4
        Using cached setuptools_scm-3.4.3.tar.gz (45 kB)
      ERROR: Exception:
      Traceback (most recent call last):
        File "/Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 186, in _main
          status = self.run(options, args)
        File "/Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 331, in run
          resolver.resolve(requirement_set)
        File "/Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 177, in resolve
          discovered_reqs.extend(self._resolve_one(requirement_set, req))
        File "/Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 333, in _resolve_one
          abstract_dist = self._get_abstract_dist_for(req_to_install)
        File "/Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 282, in _get_abstract_dist_for
          abstract_dist = self.preparer.prepare_linked_requirement(req)
        File "/Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 515, in prepare_linked_requirement
          abstract_dist = _get_prepared_distribution(
        File "/Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 94, in _get_prepared_distribution
          with req_tracker.track(req):
        File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/contextlib.py", line 113, in __enter__
          return next(self.gen)
        File "/Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/lib/python3.8/site-packages/pip/_internal/req/req_tracker.py", line 148, in track
          self.add(req)
        File "/Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/lib/python3.8/site-packages/pip/_internal/req/req_tracker.py", line 115, in add
          raise LookupError(message)
      LookupError: https://files.pythonhosted.org/packages/fe/bd/bc2fe0b14ce234bb5e2af5f3b574c5a8ef1b7845bfa41e7cf69a78627ec8/setuptools_scm-3.4.3.tar.gz#sha256=26b8a108783cd88f4b15ff1f0f347d6b476db25d0c226159b835d713f9487320 (from https://pypi.org/simple/setuptools-scm/) (requires-python:>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*) is already being built: setuptools_scm[toml]>=3.4 from https://files.pythonhosted.org/packages/fe/bd/bc2fe0b14ce234bb5e2af5f3b574c5a8ef1b7845bfa41e7cf69a78627ec8/setuptools_scm-3.4.3.tar.gz#sha256=26b8a108783cd88f4b15ff1f0f347d6b476db25d0c226159b835d713f9487320
      ----------------------------------------
    ERROR: Command errored out with exit status 2: /Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/bin/python3.8 /Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /private/var/folders/y5/x7pvzk651c3dqkllbxd1jd280000gn/T/pip-build-env-6fxxxyn6/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple --pre -- 'setuptools >= 42' 'setuptools_scm[toml]>=3.4' Check the logs for full command output.
    ----------------------------------------
  ERROR: Command errored out with exit status 1: /Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/bin/python3.8 /Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /private/var/folders/y5/x7pvzk651c3dqkllbxd1jd280000gn/T/pip-build-env-v8z5aszd/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple --pre -- 'setuptools>=34.4' wheel Check the logs for full command output.
  ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/bin/python3.8 /Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /private/var/folders/y5/x7pvzk651c3dqkllbxd1jd280000gn/T/pip-build-env-eewzron7/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple --pre -- 'setuptools >= 42' 'setuptools_scm[toml]>=3.4' Check the logs for full command output.

EDIT: Sorry, not install_requires, setup_requires

wheel's pyproject.toml:

[build-system]
requires = ["setuptools >= 42", "setuptools_scm[toml]>=3.4"]
build-backend = "setuptools.build_meta"

[tool.setuptools_scm]
write_to = "src/wheel/_version.py"
version_scheme = "post-release"

setuptools_scm's pyproject.toml:

[build-system]
requires = ["setuptools>=34.4", "wheel"]
build-backend = "setuptools.build_meta"

agronholm · 2020-01-27T14:35:13Z

I myself am trying to debug pip to understand what is going wrong. It's as if wheel is somehow requiring itself during the build, but I don't quite see how. If you could delve into pip code to help me with this, either in vivo or in vitro, that could speed up the process.

agronholm · 2020-01-27T14:36:05Z

The install_requires that wheel has on setuptools_scm[toml] is leading to trying to install wheel, because setuptools_scm has an install_requires on wheel

Damn, that would explain things.

agronholm · 2020-01-27T14:40:02Z

It's not in install_requires for setuptools_scm, though, so why would pip try to install its build requirements?

jamadden · 2020-01-27T14:41:06Z

It's not in install_requires for setuptools_scm, though, so why would pip try to install its build requirements?

Sorry, I edited my post to clarify that. It's in the build requirements in pyproject.toml (essentially setup_requires) for setuptools_scm.

agronholm · 2020-01-27T14:42:11Z

Yes, but those are only installed if the project is being built from source, which in this case should not be.

agronholm · 2020-01-27T14:43:23Z

To clarify, I am getting a similar error even when specifying only --no-binary=wheel. Thus only wheel itself should be installed from the sdist in that case.

dyeager-recurve · 2020-01-27T14:53:57Z

I get the same error with /usr/local/bin/python -m pip download --dest /tmp/wheel-error/ wheel==0.34.0 --exists-action i --no-binary :all:

Collecting wheel==0.34.0
  Using cached wheel-0.34.0.tar.gz (60 kB)
  Installing build dependencies ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-x427jk6u/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'setuptools >= 42' 'setuptools_scm[toml]>=3.4'
       cwd: None
  Complete output (39 lines):
  Collecting setuptools>=42
    Using cached setuptools-45.1.0.zip (859 kB)
  Collecting setuptools_scm[toml]>=3.4
    Using cached setuptools_scm-3.4.3.tar.gz (45 kB)
    Installing build dependencies: started
    Installing build dependencies: finished with status 'error'
    ERROR: Command errored out with exit status 2:
     command: /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-f7wcywhz/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=34.4' wheel
         cwd: None
    Complete output (27 lines):
    Collecting setuptools>=34.4
      Using cached setuptools-45.1.0.zip (859 kB)
    Collecting wheel
      Using cached wheel-0.34.0.tar.gz (60 kB)
    ERROR: Exception:
    Traceback (most recent call last):
      File "/usr/local/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 186, in _main
        status = self.run(options, args)
      File "/usr/local/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 331, in run
        resolver.resolve(requirement_set)
      File "/usr/local/lib/python3.7/site-packages/pip/_internal/legacy_resolve.py", line 177, in resolve
        discovered_reqs.extend(self._resolve_one(requirement_set, req))
      File "/usr/local/lib/python3.7/site-packages/pip/_internal/legacy_resolve.py", line 333, in _resolve_one
        abstract_dist = self._get_abstract_dist_for(req_to_install)
      File "/usr/local/lib/python3.7/site-packages/pip/_internal/legacy_resolve.py", line 282, in _get_abstract_dist_for
        abstract_dist = self.preparer.prepare_linked_requirement(req)
      File "/usr/local/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 516, in prepare_linked_requirement
        req, self.req_tracker, self.finder, self.build_isolation,
      File "/usr/local/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 94, in _get_prepared_distribution
        with req_tracker.track(req):
      File "/usr/local/lib/python3.7/contextlib.py", line 112, in __enter__
        return next(self.gen)
      File "/usr/local/lib/python3.7/site-packages/pip/_internal/req/req_tracker.py", line 148, in track
        self.add(req)
      File "/usr/local/lib/python3.7/site-packages/pip/_internal/req/req_tracker.py", line 115, in add
        raise LookupError(message)
    LookupError: https://files.pythonhosted.org/packages/15/39/728ccb81172c73f14084948b2d4561d5e51649dcf4b21d1f207b7363b1d7/wheel-0.34.0.tar.gz#sha256=a69231973dccde8558f80f2ef058335a1d0bc1b55b2a5a2dfb5097e151e3c498 (from https://pypi.org/simple/wheel/) (requires-python:!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7) is already being built: wheel==0.34.0 from https://files.pythonhosted.org/packages/15/39/728ccb81172c73f14084948b2d4561d5e51649dcf4b21d1f207b7363b1d7/wheel-0.34.0.tar.gz#sha256=a69231973dccde8558f80f2ef058335a1d0bc1b55b2a5a2dfb5097e151e3c498
    ----------------------------------------
  ERROR: Command errored out with exit status 2: /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-f7wcywhz/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=34.4' wheel Check the logs for full command output.
  ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-x427jk6u/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'setuptools >= 42' 'setuptools_scm[toml]>=3.4' Check the logs for full command output.

The context for discovery here is that our Apache Beam/Dataflow jobs first run a pip download of what is needed and started failing this morning. Hopefully this helps.

dyeager-recurve · 2020-01-27T14:57:01Z

For more context - the issue above was traced when trying to do a pip download -r requirements.txt and suddenly python-dateutil install was breaking, so I suspect this may be impacting several libraries

agronholm · 2020-01-27T15:10:03Z

This is mystifying:

pip install --user wheel==0.34.0 --no-binary=wheel
Collecting wheel==0.34.0
  Using cached wheel-0.34.0.tar.gz (60 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... error

Doesn't look like setuptools_scm's build dependency is getting in the way at all. But where are the "backend dependencies" coming from?

jamadden · 2020-01-27T15:10:14Z

To clarify, I am getting a similar error even when specifying only --no-binary=wheel. Thus only wheel itself should be installed from the sdist in that case.

Oh. In that case, I think it's because pip wants to install a project that has a pyproject.toml by…making a wheel of it. Or, more directly, it's because the setuptools build backend implementation of get_requires_for_build_wheel includes wheel in the return value. Here's a portion of the output from the above command, instrumented to print that value:

  Successfully installed setuptools-45.1.0 setuptools-scm-3.4.3 toml-0.10.0
  Cleaning up...
  Removed build tracker: '/private/var/folders/y5/x7pvzk651c3dqkllbxd1jd280000gn/T/pip-req-tracker-n3eh82a_'
  Installing build dependencies ... done
  Running command //bin/python3.8 ///pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel //T/tmpig291pld
***
Requirements: ['wheel', 'setuptools_scm>=3.4']
****
  Getting requirements to build wheel ... done
  Running command //bin/python3.8 //site-packages/pip install --ignore-installed --no-user --prefix //T/pip-build-env-5b5fcnl5/normal --no-warn-script-location -v --no-binary wheel --only-binary :none: -i https://pypi.org/simple --pre -- wheel
  Non-user install by explicit request
  Created temporary directory: /T/pip-ephem-wheel-cache-yg_vs_tf
  Created build tracker: /T/pip-req-tracker-n3eh82a_
  Entered build tracker: //T/pip-req-tracker-n3eh82a_
  Created temporary directory: //T/pip-install-itmu_l3v
  1 location(s) to search for versions of wheel:
  * https://pypi.org/simple/wheel/
  Fetching project page and analyzing links: https://pypi.org/simple/wheel/
   Collecting wheel
      Created temporary directory: /private/var/folders/y5/x7pvzk651c3dqkllbxd1jd280000gn/T/pip-unpack-887ms133
      Getting credentials from keyring for files.pythonhosted.org
      Looking up "https://files.pythonhosted.org/packages/15/39/728ccb81172c73f14084948b2d4561d5e51649dcf4b21d1f207b7363b1d7/wheel-0.34.0.tar.gz" in the cache
      Current age based on date: 2435
      Ignoring unknown cache-control directive: immutable
      Freshness lifetime from max-age: 365000000
      The response is "fresh", returning cached response
      365000000 > 2435
      Using cached wheel-0.34.0.tar.gz (60 kB)
    Cleaning up...
      Removing source in /private/var/folders/y5/x7pvzk651c3dqkllbxd1jd280000gn/T/pip-install-0h5ui0t6/wheel
    Removed build tracker: '/private/var/folders/y5/x7pvzk651c3dqkllbxd1jd280000gn/T/pip-req-tracker-n3eh82a_'
    ERROR: Exception:
    Traceback (most recent call last):
      File "/Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 186, in _main
        status = self.run(options, args)
      File "/Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 331, in run
        resolver.resolve(requirement_set)
      File "/Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 177, in resolve
        discovered_reqs.extend(self._resolve_one(requirement_set, req))
      File "/Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 333, in _resolve_one
        abstract_dist = self._get_abstract_dist_for(req_to_install)
      File "/Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 282, in _get_abstract_dist_for
        abstract_dist = self.preparer.prepare_linked_requirement(req)
      File "/Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 515, in prepare_linked_requirement
        abstract_dist = _get_prepared_distribution(
      File "/Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 94, in _get_prepared_distribution
        with req_tracker.track(req):
      File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/contextlib.py", line 113, in __enter__
        return next(self.gen)
      File "/Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/lib/python3.8/site-packages/pip/_internal/req/req_tracker.py", line 148, in track
        self.add(req)
      File "/Users/jmadden/Projects/VirtualEnvs/tmp-b5481b392b4dda0c/lib/python3.8/site-packages/pip/_internal/req/req_tracker.py", line 115, in add
        raise LookupError(message)
    LookupError: https://files.pythonhosted.org/packages/15/39/728ccb81172c73f14084948b2d4561d5e51649dcf4b21d1f207b7363b1d7/wheel-0.34.0.tar.gz#sha256=a69231973dccde8558f80f2ef058335a1d0bc1b55b2a5a2dfb5097e151e3c498 (from https://pypi.org/simple/wheel/) (requires-python:!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7) is already being built: wheel from https://files.pythonhosted.org/packages/15/39/728ccb81172c73f14084948b2d4561d5e51649dcf4b21d1f207b7363b1d7/wheel-0.34.0.tar.gz#sha256=a69231973dccde8558f80f2ef058335a1d0bc1b55b2a5a2dfb5097e151e3c498

agronholm · 2020-01-27T15:14:59Z

@ncoghlan If this is true, does it prevent the wheel project from using isolated builds, by design?

r0bnet · 2020-01-27T15:19:45Z

We're facing a similar issue with pyspark==2.4.4 installed in a freshly created virtual environment. It doesn't affect us when we install wheel (even 0.34.0) before trying to install pyspark.

Docker image: Ubuntu 16.04.6 LTS

Error:

Collecting pyspark==2.4.4
  Downloading pyspark-2.4.4.tar.gz (215.7 MB)
    ERROR: Command errored out with exit status 1:
     command: /home/agentadmin/workspace/_bugfix_DATAENG-1093-performance/.venv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-uam4drd4/pyspark/setup.py'"'"'; __file__='"'"'/tmp/pip-install-uam4drd4/pyspark/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-uam4drd4/pyspark/pip-egg-info
         cwd: /tmp/pip-install-uam4drd4/pyspark/
    Complete output (45 lines):
    Could not import pypandoc - required to package PySpark
    Maybe try:
    
        sudo apt-get install pandoc
    See http://johnmacfarlane.net/pandoc/installing.html
    for installation options
    ---------------------------------------------------------------
    
    zip_safe flag not set; analyzing archive contents...
    pypandoc.__pycache__.__init__.cpython-35: module references __file__
    
    
    !!! pandoc not found, long_description is bad, don't upload this to PyPI !!!
    
    
    
    Installed /tmp/pip-install-uam4drd4/pyspark/.eggs/pypandoc-1.4-py3.5.egg
    Searching for wheel>=0.25.0
    Reading https://pypi.python.org/simple/wheel/
    Best match: wheel 0.34.0
    Downloading https://files.pythonhosted.org/packages/15/39/728ccb81172c73f14084948b2d4561d5e51649dcf4b21d1f207b7363b1d7/wheel-0.34.0.tar.gz#sha256=a69231973dccde8558f80f2ef058335a1d0bc1b55b2a5a2dfb5097e151e3c498
    Processing wheel-0.34.0.tar.gz
    Writing /tmp/easy_install-r6tqgyee/wheel-0.34.0/setup.cfg
    Running wheel-0.34.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-r6tqgyee/wheel-0.34.0/egg-dist-tmp-9vm8xt1i
    /usr/lib/python3.5/distutils/dist.py:261: UserWarning: Unknown distribution option: 'use_scm_version'
      warnings.warn(msg)
    warning: install_lib: 'build/lib' does not exist -- no Python modules to install
    
    zip_safe flag not set; analyzing archive contents...
    Moving UNKNOWN-0.0.0-py3.5.egg to /tmp/pip-install-uam4drd4/pyspark/.eggs
    
    Installed /tmp/pip-install-uam4drd4/pyspark/.eggs/UNKNOWN-0.0.0-py3.5.egg
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-uam4drd4/pyspark/setup.py", line 224, in <module>
        'Programming Language :: Python :: Implementation :: PyPy']
      File "/usr/lib/python3.5/distutils/core.py", line 108, in setup
        _setup_distribution = dist = klass(attrs)
      File "/home/agentadmin/workspace/_bugfix_DATAENG-1093-performance/.venv/lib/python3.5/site-packages/setuptools/dist.py", line 269, in __init__
        self.fetch_build_eggs(attrs['setup_requires'])
      File "/home/agentadmin/workspace/_bugfix_DATAENG-1093-performance/.venv/lib/python3.5/site-packages/setuptools/dist.py", line 313, in fetch_build_eggs
        replace_conflicting=True,
      File "/home/agentadmin/workspace/_bugfix_DATAENG-1093-performance/.venv/lib/python3.5/site-packages/pkg_resources/__init__.py", line 829, in resolve
        raise DistributionNotFound(req, requirers)
    pkg_resources.DistributionNotFound: The 'wheel>=0.25.0' distribution was not found and is required by pypandoc
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
Makefile:11: recipe for target 'install' failed

We'd like to not install wheel manually in every of our projects and this is just a workaround for us. With v0.33.6 everything was fine.

pfmoore · 2020-01-27T15:22:05Z

It's not by design, but it may be an unintended oversight. Support for in-tree backends was added to PEP 517 not that long ago, to address similar issues with bootstrapping setuptools - but pip doesn't have support for that addition to the PEP yet (as far as I am aware).

But nothing has changed in pip recently that would have affected this, so what's changed? If we can confirm what changed, we might be able to understand better why this is happening.

agronholm · 2020-01-27T15:22:57Z

@r0bnet The traceback doesn't seem to point to wheel being the real problem there. I mean, why is wheel not found if it's declared as a build dependency?

jamadden · 2020-01-27T15:23:10Z

But nothing has changed in pip recently that would have affected this, so what's changed? If we can confirm what changed, we might be able to understand better why this is happening.

Wheel 0.34 is the first to distribute an sdist including pyproject.toml

agronholm · 2020-01-27T15:25:03Z

What's even more mysterious is that installing from a clean local source tree (pip install --user ~/checkout/wheel) works fine. The process should be the same, so what gives?

EDIT: never mind, with --no-binary=wheel that fails too.

agronholm · 2020-01-27T15:31:53Z

So I suppose that my only option here is to give up on PEP 517 support? I just tried removing setuptools_scm as a build dependency but I still got the same error, which makes sense if pip is trying to install wheel to build wheel.

pfmoore · 2020-01-27T15:40:56Z

@jamadden Ah yes, that might do it 🙁

So basically, by distributing a pyproject.toml, the code is saying "in order to build a wheel for this project, you need to create a build environment containing the build backend, and then run it on the project". But the build backend is setuptools + wheel, and so you need to install wheel in the build environment. Hence the loop.

Also relevant may be the --no-binary flag. I think we'll install wheel from a wheel when setting up the build backend, but obviously if you say --no-binary you prohibit that (there was a discussion about allowing wheels for the build backend even when --no-binary was specified, but you'd need to read the thread on Discourse rather than relying on my memory for that much detail 🙂 ) I'm not clear from the discussion whether the problem happens without --no-binary.

That's what the in-tree backend feature is intended to handle.

@agronholm You should probably check with setuptools (ping @jaraco @pganssle) as they have been in this situation for a while now. But yes, short term you may not be able to use PEP 517 to publish wheels. Medium term, you should look at the in-tree backend support and confirm if it addresses your need - if it doesn't, we may have to rethink - it was definitely intended that it would handle issues like this (but we only really had setuptools and flit as examples at the time, so input from the wheel project would be really useful).

Fixes #331. Fixes #332. Fixes #333.

agronholm · 2020-01-27T16:36:57Z

I've created a new branch with the fix. I have to go for a while now but feel free to review it.

agronholm · 2020-01-27T16:41:16Z

This is in #334.

dyeager-recurve · 2020-01-27T17:48:08Z

Thanks for the quick turnaround @agronholm !

agronholm · 2020-01-27T20:30:52Z

I've released v0.34.1 now.

jamadden mentioned this issue Jan 27, 2020

Wheel 0.34.0 broke installation from sdist: LookupError #333

Closed

agronholm added a commit that referenced this issue Jan 27, 2020

Fixed installation from sdist not working

79e3ec4

Fixes #331. Fixes #332. Fixes #333.

agronholm closed this as completed in 510a531 Jan 27, 2020

agronholm mentioned this issue Apr 14, 2020

implicit dependency on setuptools causes build failures on pip #344

Closed

jaraco mentioned this issue Apr 19, 2020

Bootstrapping problem (how to bootstrap all-source environments) pypa/packaging-problems#342

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Wheel 0.34 broke installing in docker-alpine #332

Wheel 0.34 broke installing in docker-alpine #332

andronov04 commented Jan 27, 2020

LeonJansen commented Jan 27, 2020

agronholm commented Jan 27, 2020

tomasmor42 commented Jan 27, 2020

jamadden commented Jan 27, 2020 •

edited

Loading

agronholm commented Jan 27, 2020

agronholm commented Jan 27, 2020

agronholm commented Jan 27, 2020

jamadden commented Jan 27, 2020

agronholm commented Jan 27, 2020

agronholm commented Jan 27, 2020

dyeager-recurve commented Jan 27, 2020

dyeager-recurve commented Jan 27, 2020 •

edited

Loading

agronholm commented Jan 27, 2020

jamadden commented Jan 27, 2020 •

edited

Loading

agronholm commented Jan 27, 2020

r0bnet commented Jan 27, 2020 •

edited

Loading

pfmoore commented Jan 27, 2020

agronholm commented Jan 27, 2020

jamadden commented Jan 27, 2020

agronholm commented Jan 27, 2020 •

edited

Loading

agronholm commented Jan 27, 2020

pfmoore commented Jan 27, 2020 •

edited

Loading

agronholm commented Jan 27, 2020

agronholm commented Jan 27, 2020

dyeager-recurve commented Jan 27, 2020

agronholm commented Jan 27, 2020

Wheel 0.34 broke installing in docker-alpine #332

Wheel 0.34 broke installing in docker-alpine #332

Comments

andronov04 commented Jan 27, 2020

LeonJansen commented Jan 27, 2020

agronholm commented Jan 27, 2020

tomasmor42 commented Jan 27, 2020

jamadden commented Jan 27, 2020 • edited Loading

agronholm commented Jan 27, 2020

agronholm commented Jan 27, 2020

agronholm commented Jan 27, 2020

jamadden commented Jan 27, 2020

agronholm commented Jan 27, 2020

agronholm commented Jan 27, 2020

dyeager-recurve commented Jan 27, 2020

dyeager-recurve commented Jan 27, 2020 • edited Loading

agronholm commented Jan 27, 2020

jamadden commented Jan 27, 2020 • edited Loading

agronholm commented Jan 27, 2020

r0bnet commented Jan 27, 2020 • edited Loading

pfmoore commented Jan 27, 2020

agronholm commented Jan 27, 2020

jamadden commented Jan 27, 2020

agronholm commented Jan 27, 2020 • edited Loading

agronholm commented Jan 27, 2020

pfmoore commented Jan 27, 2020 • edited Loading

agronholm commented Jan 27, 2020

agronholm commented Jan 27, 2020

dyeager-recurve commented Jan 27, 2020

agronholm commented Jan 27, 2020

jamadden commented Jan 27, 2020 •

edited

Loading

dyeager-recurve commented Jan 27, 2020 •

edited

Loading

jamadden commented Jan 27, 2020 •

edited

Loading

r0bnet commented Jan 27, 2020 •

edited

Loading

agronholm commented Jan 27, 2020 •

edited

Loading

pfmoore commented Jan 27, 2020 •

edited

Loading