Unable to pip-compile textract

[seankfh]

I'm unable to pip-compile textract (see error below). Is there anything I can do to pass textract through to the output file without processing it or avoid this another way?

Environment Versions

1. Debian 11.
2. Python version: 3.9.2
3. pip version: 23.0.1
4. pip-tools version: 6.12.3

Steps to replicate

1. Add textract==1.6.4 to input file.
2. Run pip-compile.
3. See error message.

Expected result

textract and related packages in generated output file.

Actual result

Process exits with error:

    error: subprocess-exited-with-error
    
    × python setup.py egg_info did not run successfully.
    │ exit code: 1
    ╰─> [3 lines of output]
        error in textract setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers; Expected end or semicolon (after version specifier)
            extract-msg<=0.29.*
                       ~~~~~~^
        [end of output]

[atugushev]

Couldn't confirm locally. Could you please follow the issue template for the bug report?

[seankfh]

Thanks for looking @atugushev I wasn't sure which of the two template options to follow. This is done.

[atugushev]

Successfully compiled on:

root@py39:/# uname -a
Linux py39 5.10.47-linuxkit #1 SMP Sat Jul 3 21:51:47 UTC 2021 x86_64 GNU/Linux

root@py39:/# pip-compile  --version
pip-compile, version 6.12.3

root@py39:/# pip --version
pip 23.0.1 

pip-compile

root@py39:/# echo textract==1.6.4 | pip-compile - -qo-
WARNING: the legacy dependency resolver is deprecated and will be removed in future versions of pip-tools. The default resolver will be changed to 'backtracking' in pip-tools 7.0.0. Specify --resolver=backtracking to silence this warning.
#
# This file is autogenerated by pip-compile with Python 3.9
# by the following command:
#
#    pip-compile --output-file=- -
#
argcomplete==1.10.3
    # via textract
beautifulsoup4==4.8.2
    # via textract
chardet==3.0.4
    # via
    #   pdfminer-six
    #   textract
compressed-rtf==1.0.6
    # via extract-msg
docx2txt==0.8
    # via textract
ebcdic==1.1.1
    # via extract-msg
ebooklib==0.18
    # via textract
extract-msg==0.28.7
    # via textract
imapclient==2.1.0
    # via extract-msg
lxml==4.9.2
    # via
    #   ebooklib
    #   python-pptx
olefile==0.46
    # via extract-msg
pdfminer-six==20191110
    # via textract
pillow==9.4.0
    # via python-pptx
pycryptodome==3.17
    # via pdfminer-six
python-pptx==0.6.21
    # via textract
pytz-deprecation-shim==0.1.0.post0
    # via tzlocal
six==1.12.0
    # via
    #   ebooklib
    #   imapclient
    #   pdfminer-six
    #   textract
sortedcontainers==2.4.0
    # via pdfminer-six
soupsieve==2.4
    # via beautifulsoup4
speechrecognition==3.8.1
    # via textract
textract==1.6.4
    # via -r -
tzdata==2022.7
    # via pytz-deprecation-shim
tzlocal==4.3
    # via extract-msg
xlrd==1.2.0
    # via textract
xlsxwriter==3.0.9
    # via python-pptx

[seankfh]

Seems to be related to _disutils_hack, but I have tried updating setuptools and saw no change:

~/projects/forms$ echo textract==1.6.4 | pip-compile - -qo-
~/projects/forms-env/lib/python3.9/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
WARNING: the legacy dependency resolver is deprecated and will be removed in future versions of pip-tools. The default resolver will be changed to 'backtracking' in pip-tools 7.0.0. Specify --resolver=backtracking to silence this warning.
    error: subprocess-exited-with-error

    × python setup.py egg_info did not run successfully.
    │ exit code: 1
    ╰─> [3 lines of output]
        error in textract setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers; Expected end or semicolon (after version specifier)
            extract-msg<=0.29.*
                       ~~~~~~^
        [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
Traceback (most recent call last):
  File "~/projects/forms-env/lib/python3.9/site-packages/pip/_internal/operations/build/metadata_legacy.py", line 64, in generate_metadata
    call_subprocess(
  File "~/projects/forms-env/lib/python3.9/site-packages/pip/_internal/utils/subprocess.py", line 224, in call_subprocess
    raise error
pip._internal.exceptions.InstallationSubprocessError: python setup.py egg_info exited with 1

[seankfh]

Executing export SETUPTOOLS_USE_DISTUTILS=stdlib from https://setuptools.pypa.io/en/latest/deprecated/distutils-legacy.html eliminated the distutils warning but the rest of the error messaged remained.

[seankfh]

Thanks everyone for your patience and guidance. I believe my local issue was solved by uninstalling / reinstalling pip:

deactivate
rm -rf env
python3 -m pip uninstall pip
python3 -m pip install pip
python3 -m venv env
source env/bin/activate
pip install pip-tools
echo textract==1.6.4 | pip-compile - -qo-

[seankfh]

Actually I am getting the same error as I was with pip-compile running pip-sync now. Also this time I was able to recreate it in Debian and on a Mac.

echo textract==1.6.4 > requirements.in
pip-compile -o requirements.txt requirements.in
pip-sync requirements.txt

Yields "ERROR: Failed building wheel for textract".

[AndydeCleyre]

Is the problem worked around if you have wheel installed first?

[seankfh]

It doesn't seem to make a difference whether wheel is installed first.

[AndydeCleyre]

Thanks, I reproduced in a python:3.9 container, and could do so reliably by alternating runs of pip-sync and pip-sync /dev/null. Then I uninstalled wheel and it built successfully, warning that

DEPRECATION: textract is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed...
...

And it syncs without issue if I pip install 'wheel<0.40.0'.

Looking over at pypa/wheel#520, it seems

.* suffix can only be used with == or != operators

whereas textract uses it with <=.

[seankfh]

I tried uninstalling wheel and then running pip install 'wheel<0.40.0' but still received the same error with wheel-0.38.4 and other older versions. Good find on the specific wheel issue. It really worked for you with <0.40.0?

[AndydeCleyre]

Yeah, here's what I did:

$ podman run -it --rm --net=host docker.io/library/python:3.9 bash
root@pop-os:/# python -m venv venv
root@pop-os:/# . ./venv/bin/activate
(venv) root@pop-os:/# pip install -U pip pip-tools 'wheel<0.40.0'
(venv) root@pop-os:/# pip list
Package         Version
--------------- -------
build           0.10.0
click           8.1.3
packaging       23.0
pip             23.0.1
pip-tools       6.12.3
pyproject_hooks 1.0.0
setuptools      58.1.0
tomli           2.0.1
wheel           0.38.4

(venv) root@pop-os:/# echo 'textract==1.6.4' >requirements.in
(venv) root@pop-os:/# pip-compile
(venv) root@pop-os:/# pip-sync

(venv) root@pop-os:/# pip-sync /dev/null
(venv) root@pop-os:/# rm requirements.txt

(venv) root@pop-os:/# pip-compile --resolver=backtracking
(venv) root@pop-os:/# pip-sync

That all works fine. This seems to be an issue of textract using an invalid requirement, and wheel/packaging now being more strict about it.

[seankfh]

Thank you so much. I'm still working to get crystal clear on which step you listed above makes this work correctly in my environment. Even after using Python for so long apparently I have a lot to learn about packaging, setuptools, and wheel.

[atugushev]

Welcome to the Python Packaging world ^^

[AndydeCleyre]

I think since this is down to a bad version specifier in textract, and an issue is now up in their tracker (thanks @seankfh), we can close this here.