installation is broken on MacOS ?

[neutrinoceros]

Describe the bug

Hi ! I attempted to install pipx following the instructions on Pypi

How to reproduce

brew install pipx
pipx ensurepath

output

Traceback (most recent call last):
  File "/usr/local/Cellar/pipx/0.15.4.0/libexec/bin/pipx", line 6, in <module>
    from pkg_resources import load_entry_point
ModuleNotFoundError: No module named 'pkg_resources'

Expected behavior
Successful install ? :)

my specs

macOS version Catalina 10.15.6
Homebrew 2.4.9

I'm happy to provide more specific details about my env if needed !

[neutrinoceros]

I think this comes down to the fact that the python version homebrew tried to install as a dependency for pipx failed to link. I'll keep looking

[cs01]

@AlJohri does the brew formula need updating?

[itsayellow]

brew install pipx and pipx ensurepath Works For Me on
macOS Catalina 10.15.6, Homebrew 2.4.9-317-g02969cd

[itsayellow]

@neutrinoceros homebrew recently (last couple of weeks) moved from python 3.7 as default to python 3.8 as default. Could that contribute to your environment's trouble?

[neutrinoceros]

Possibly... I reported this a bit early before I realised that pkg_resources is part of the standard library. I was since able to check that indeed, the python runtime that was installed on my machine as a side effect of brew install pipx can't import pkg_resources at all, so now I believe it's more likely that the brew recipe for python itself is broken.

[uranusjr]

Nitpicking, pkg_resources is not part of the standard library, it’s a third-party library (part of setuptools) that gets installed by default when Python is installed (that’s a mouthful).

[AlJohri]

The homebrew formula is up to date and pipx ensurepath is also working for me.

@neutrinoceros can you perhaps re-install your homebrew python and see if that fixes the issue? Something along the lines of:

brew uninstall pipx
brew uninstall python3
brew update
brew install python3
brew install pipx
pipx ensurepath

[neutrinoceros]

Sorry for not reporting in more detail but I tried this already and it leads to the same result. I also tried installing a different version of python with Homebrew but Python 3.8.5 was still installed and used after that.
For reference I was able to install pipx via pip, so I'm fine with this being closed if we can't figure out what is different/broken on my machine :)

[anthem-bmk]

Why is brew install pipx recommended over python3 -m pip install pipx in the first place? I tried the latter on macOS before reading the docs and things seem to be working fine with python 3.7.7

[itsayellow]

Why is brew install pipx recommended over python3 -m pip install pipx in the first place? I tried the latter on macOS before reading the docs and things seem to be working fine with python 3.7.7

@AlJohri knows more than I do about this, but among the advantages: brew install pipx installs pipx and its dependencies to its own isolated environment, so it doesn't clutter up your main python installed packages (e.g. when you type pip list).

[gaborbernat]

python3 -m pip install pipx this can work just fine most of the time, fail horribly if the system python is read-only, cause random issues if pipx requires dependencies that are in conflict with other packages you've installed. We have not solved yet how to bootstrap pipx itself (something like curl https://get-pipx.zipapp && python3 get-pipx.zipapp to install pipx). Until we do using a operating system installer is your safest option.

[uranusjr]

There’s another issue if your pip or python -m pip is tied to Homebrew. Homebrew automatically removes “outdated” installations, so pipx would have the interpreter it installed against pulled from under when you upgrade Python from (say) 3.8 to 3.9. brew install pipx handles that transition for you. (Personally I think Homebrew’s philosophy in this area is very questionable, but that’s another topic altogether and wouldn’t help here.)

[layday]

This is presumably an issue with the new Nix package as well - pipx packages will not be protected from garbage collection so if a pipx update links to a new Python executable, well.... (This'll probably come up a lot more often than it does with brew too cuz of Nix's functional nature - any time a transitive dependency changes the dependency graph changes and when the dependency graph changes its hash is regenerated and guess what - the hash is included in the path.)

[1oglop1]

I have a bad experience with homebrew dependency system and brewed python in general and I'm thinking that the best option for tool like pipx would be to vendor python in it's installation, which would provide maximal independence from environments and other python installations. Another benefit would be potential option to let pipx maintain multiple versions of python and tools as a future enhancements.

The only downside would probably be a slight decrease of cross-platform compatibility but I'm not aware of other potential problems.

What do you think @cs01 would this be something you consider for the project?

[uranusjr]

The Scoop package (for Windows) currently does just that, and a patch that made it work was recently merged into the main pipx (yay). Currently the biggest hurdle to make this work is that it’s significantly more difficult to create a “portable” Python to bundle with pipx on POSIX systems. I know there are projects doing this (pyoxidizer comes to mind), but someone will need to invest time and work to make sure it’s viable.

[cs01]

The Scoop package (for Windows) currently does just that

Do all scoop packages work like this, or was something done specifically for pipx? How would something like this work for homebrew?

Tbh I don't have a good understanding of how Python installations/homebrew work. My impression is this is not possible, since pyenv would probably be doing that. But it is not -- it builds from source instead.

[uranusjr]

pyenv needs to provide a fully operational Pyehon interepreter because it cannot control how the user would use the interpreters it installs, and indeed there’s currently not a way to guarentee that except building from source. But pipx does not have that constraint; the interpreter it runs on only needs to be able to run straight-up Python modules, and create virtual environments (and even that would become optional with #448, or if #465 is extended for non-Windows platforms). Therer are many portable possibilities if you remove the many edge cases in the Python interpreter that have nothing to do with pipx.

[cs01]

pyenv needs to provide a fully operational Python interepreter because it cannot control how the user would use the interpreters it installs

the [pipx] interpreter it runs on only needs to be able to run straight-up Python modules

What are the differences between a fully operational interpreter and what pipx would need?

[cs01]

What do you think @cs01 would this be something you consider for the project?

To answer your question I am open to discussing this, but I do not have the expertise or bandwidth to take this on. It would have to be championed by someone else, similar to how @AlJohri championed pipx on homebrew and @uranusjr did for Scoop.

[AlJohri]

I have a bad experience with homebrew dependency system and brewed python in general and I'm thinking that the best option for tool like pipx would be to vendor python in it's installation, which would provide maximal independence from environments and other python installations.

This is also something I'm pretty interested in @1oglop1. You can read more in this issue here: #244

[uranusjr]

What are the differences between a fully operational interpreter and what pipx would need?

The most significant feature that pipx does not need is to pip install stuff into the system location. Python hard-codes the installation prefix (e.g. /usr) during compilation, and the prefix is used to e.g. generate shebangs, locate the site-packages directory. pipx can avoid that as long as it does not use sys.executable to install packages (which is why #465 was needed to make embedded distribution work).

Another significant difficulty faced by portable installations is readline support. This is very important for the interactive shell (on POSIX systems), but pipx does not need that at all since the underlying python command is never expected to be invoked directly, only as a vehicle to run the pipx module.

Do all scoop packages work like this, or was something done specifically for pipx? How would something like this work for homebrew?

There are not many Python-based tools on Scoop, but vendoring is a well-established practice on Windows in general. Docker on Windows ships a portable Python interpreter for docker-compose, instead of depending on a system-wide Python installation, for example.

The same practice is actually quite popular on Mac as well among GUI developers (who mainly distribute .app bundles). Docker for Mac also ships its own Python interepreter IIUC. But Homebrew distributors follow a different methodology that mirrors more to Unix-land, where applications prefer to depend on other system-wide stuff. Both methods have their own advantages, but the standalone, bundling approach works better for pipx specifically IMO.

[syrofoam]

Same problem on Archlinux python 3.9.0 ModuleNotFoundError: No module named 'insert program". Tried reinstalling pipx, worked a few days ago.