Introduce a new env variable to allow spaces in home path (pypa#1446)

* Introduce a new env variable to allow spaces in home path

* Improve docs for moving the pipx installation and troubleshooting

* Add news fragment

* fix windows default location

Co-authored-by: Xuan (Sean) Hu <[email protected]>

* Translate windows bash instructions to powershell

Co-authored-by: Xuan (Sean) Hu <[email protected]>

* Add note for using different shell on windows

* make sure linux instructions work with spaces in path and bash

* add double quotes around powershell variables

* Apply suggestions from code review

Co-authored-by: chrysle <[email protected]>

* update links

---------

Co-authored-by: Xuan (Sean) Hu <[email protected]>
Co-authored-by: chrysle <[email protected]>

changelog.d/1320.bugfix

@@ -0,0 +1 @@
+Introduce `PIPX_HOME_ALLOW_SPACE` environment variable, to silence the spaces in pipx home path warning

docs/installation.md

@@ -156,8 +156,8 @@ and `~\pipx` on Windows. For compatibility reasons, if `~/.local/pipx` on Linux,
 This can be overridden with the `PIPX_HOME` environment variable.
 
 In case one of these fallback locations exist, we recommend either manually moving the pipx files to the new default location
-(see the `Troubleshooting` section of the docs), or setting the `PIPX_HOME` environment variable (discarding files existing in
-the fallback location).
+(see the [Moving your pipx installation](installation.md#moving-your-pipx-installation) section of the docs), or setting the
+`PIPX_HOME` environment variable (discarding files existing in the fallback location).
 
 As an example, you can install global apps accessible by all users on your system with the following command (on MacOS,
 Linux, and Windows WSL):
@@ -184,7 +184,7 @@ sudo PIPX_HOME=/opt/pipx PIPX_BIN_DIR=/usr/local/bin PIPX_MAN_DIR=/usr/local/sha
 > See the [platformdirs documentation](https://platformdirs.readthedocs.io/en/latest/api.html#platforms) for details.
 >
 > This was reverted in 1.5.0 for Windows and MacOS. We heavily recommend not using these locations on Windows and MacOS anymore, due to
-> multiple incompatibilities discovered with these locations, documented [here](https://github.com/pypa/pipx/discussions/1247#discussion-6188916).
+> multiple incompatibilities discovered with these locations, documented [here](troubleshooting.md#why-are-spaces-in-the-pipx_home-path-bad).
 
 ### Customising your installation
 
@@ -257,3 +257,65 @@ You can easily get your shell's tab completions working by following instruction
 ```
 pipx completions
 ```
+
+## Moving your pipx installation
+
+The below code snippets show how to move your pipx installation to a new directory.
+As an example, they move from a non-default location to the current default locations.
+If you wish to move to a different location, just replace the `NEW_LOCATION` value.
+
+### MacOS
+
+Current default location: `~/.local`
+
+```bash
+NEW_LOCATION=~/.local
+cache_dir=$(pipx environment --value PIPX_VENV_CACHEDIR)
+logs_dir=$(pipx environment --value PIPX_LOG_DIR)
+trash_dir=$(pipx environment --value PIPX_TRASH_DIR)
+home_dir=$(pipx environment --value PIPX_HOME)
+rm -rf "$cache_dir" "$logs_dir" "$trash_dir"
+mkdir -p $NEW_LOCATION && mv "$home_dir" $NEW_LOCATION
+pipx reinstall-all
+```
+
+### Linux
+
+Current default location: `~/.local/share`
+
+```bash
+cache_dir=$(pipx environment --value PIPX_VENV_CACHEDIR)
+logs_dir=$(pipx environment --value PIPX_LOG_DIR)
+trash_dir=$(pipx environment --value PIPX_TRASH_DIR)
+home_dir=$(pipx environment --value PIPX_HOME)
+# If you wish another location, replace the expression below
+# and set `NEW_LOCATION` explicitly
+NEW_LOCATION="${XDG_DATA_HOME:-$HOME/.local/share}"
+rm -rf "$cache_dir" "$logs_dir" "$trash_dir"
+mkdir -p $NEW_LOCATION && mv "$home_dir" $NEW_LOCATION
+pipx reinstall-all
+```
+
+### Windows
+
+Current default location: `~/pipx`
+
+```powershell
+$NEW_LOCATION = Join-Path "$HOME" 'pipx'
+$cache_dir = pipx environment --value PIPX_VENV_CACHEDIR
+$logs_dir = pipx environment --value PIPX_LOG_DIR
+$trash_dir = pipx environment --value PIPX_TRASH_DIR
+$home_dir = pipx environment --value PIPX_HOME
+
+Remove-Item -Recurse -Force -ErrorAction SilentlyContinue "$cache_dir", "$logs_dir", "$trash_dir"
+
+# Remove the destination directory to ensure rename behavior of `Move-Item`
+Remove-Item -Recurse -Force -ErrorAction SilentlyContinue "$NEW_LOCATION"
+
+Move-Item -Path $home_dir -Destination "$NEW_LOCATION"
+pipx reinstall-all
+```
+
+If you would prefer doing it in bash via git-bash/WSL, feel free to use
+the MacOS/Linux instructions, changing the `$NEW_LOCATION` to the Windows
+version.

docs/troubleshooting.md

@@ -148,13 +148,28 @@ In Pipx version 1.5.0, this was reverted for Windows and MacOS. It defaults agai
 
 If you have a `pipx` version later than 1.2.0 and want to migrate from the old path to the new paths, you can move the
 `~/.local/pipx` directory to the new location (after removing cache, log, and trash directories which will get recreated
-automatically) and then reinstall all packages. For example, on Linux systems, `PIPX_HOME` moves from `~/.local/pipx` to
-`~/.local/share/pipx` so you can do this:
+automatically) and then reinstall all packages.
 
-```
-rm -rf ~/.local/pipx/{.cache,logs,trash}
-mkdir -p ~/.local/share && mv ~/.local/pipx ~/.local/share/
-pipx reinstall-all
-```
+Please refer to [Installation](installation.md#moving-your-pipx-installation) on how to move it.
+
+## Warning: Found a space in the pipx home path
+
+In pipx version 1.5, we introduced the warning you're seeing, as multiple incompatibilites with spaces in the pipx home path were discovered. You may see this for the following reasons:
+
+1. From pipx version 1.3 to 1.5, we were by default using a path with a space on it on MacOS. This unfortunately means, that all users that installed pipx in this time frame and were using the default behavior are seeing this warning now.
+2. You set your `PIPX_HOME` to a path with spaces in it explicitly or because your `$HOME` path contains a space.
+
+### Why are spaces in the `PIPX_HOME` path bad
+
+The main reason we can't support paths with spaces is that shebangs don't support spaces in the interpreter path. All applications installed via `pipx` are installed via `pip`, which creates a script with a shebang at the top, defining the interpreter of the `venv` to use.
+
+`pip` does some magic to the shebang for scripts defined as a `script`, that resolves this issue. Unfortunately, many libraries define their scripts as `console_scripts`, where `pip` does not perform this logic. Therefore, these scripts cannot be run if installed with `pipx` in a path with spaces, as the path to the `venv` and therefore the interpreter to use will contain spaces.
+
+If you want to use a script installed via pipx in a shebang itself (common for example for the aws cli), you run into a similar problem, as the path to the installed script will contain a space.
+
+### How to fix
+
+You can generally fix this by using our default locations, as long as your `$HOME` path does not contain spaces.
+Please refer to our [Installation](installation.md#moving-your-pipx-installation) docs on how to move the `pipx` installation.
 
-For moving the paths back after 1.5.0, you can perform the same steps, switching the paths around.
+If you're really sure you want to stick to your path with spaces, to suppress the warning set the `PIPX_HOME_ALLOW_SPACE` environment variable to `true`.

src/pipx/commands/environment.py

@@ -17,6 +17,7 @@ def environment(value: str) -> ExitCode:
         "PIPX_DEFAULT_PYTHON",
         "PIPX_FETCH_MISSING_PYTHON",
         "USE_EMOJI",
+        "PIPX_HOME_ALLOW_SPACE",
     ]
     derived_values = {
         "PIPX_HOME": paths.ctx.home,
@@ -30,6 +31,7 @@ def environment(value: str) -> ExitCode:
         "PIPX_STANDALONE_PYTHON_CACHEDIR": paths.ctx.standalone_python_cachedir,
         "PIPX_DEFAULT_PYTHON": DEFAULT_PYTHON,
         "USE_EMOJI": str(EMOJI_SUPPORT).lower(),
+        "PIPX_HOME_ALLOW_SPACE": str(paths.ctx.allow_spaces_in_home_path).lower(),
     }
     if value is None:
         print("Environment variables (set by user):")

src/pipx/main.py

@@ -84,11 +84,12 @@ def prog_name() -> str:
 PIPX_DESCRIPTION += pipx_wrap(
     """
     optional environment variables:
-      PIPX_HOME             Overrides default pipx location. Virtual Environments will be installed to $PIPX_HOME/venvs.
-      PIPX_BIN_DIR          Overrides location of app installations. Apps are symlinked or copied here.
-      PIPX_MAN_DIR          Overrides location of manual pages installations. Manual pages are symlinked or copied here.
-      PIPX_DEFAULT_PYTHON   Overrides default python used for commands.
-      USE_EMOJI             Overrides emoji behavior. Default value varies based on platform.
+      PIPX_HOME              Overrides default pipx location. Virtual Environments will be installed to $PIPX_HOME/venvs.
+      PIPX_BIN_DIR           Overrides location of app installations. Apps are symlinked or copied here.
+      PIPX_MAN_DIR           Overrides location of manual pages installations. Manual pages are symlinked or copied here.
+      PIPX_DEFAULT_PYTHON    Overrides default python used for commands.
+      USE_EMOJI              Overrides emoji behavior. Default value varies based on platform.
+      PIPX_HOME_ALLOW_SPACE  Overrides default warning on spaces in the home path
     """,
     subsequent_indent=" " * 24,  # match the indent of argparse options
     keep_newlines=True,
@@ -906,7 +907,7 @@ def _add_environment(subparsers: argparse._SubParsersAction, shared_parser: argp
 
             Available variables:
             PIPX_HOME, PIPX_BIN_DIR, PIPX_MAN_DIR, PIPX_SHARED_LIBS, PIPX_LOCAL_VENVS,
-            PIPX_LOG_DIR, PIPX_TRASH_DIR, PIPX_VENV_CACHEDIR, PIPX_DEFAULT_PYTHON, USE_EMOJI
+            PIPX_LOG_DIR, PIPX_TRASH_DIR, PIPX_VENV_CACHEDIR, PIPX_DEFAULT_PYTHON, USE_EMOJI, PIPX_HOME_ALLOW_SPACE
             """
         ),
         parents=[shared_parser],

src/pipx/paths.py

@@ -6,7 +6,7 @@
 from platformdirs import user_cache_path, user_data_path, user_log_path
 
 from pipx.constants import LINUX, WINDOWS
-from pipx.emojis import hazard
+from pipx.emojis import hazard, strtobool
 from pipx.util import pipx_wrap
 
 if LINUX:
@@ -106,12 +106,16 @@ def make_global(self) -> None:
     def standalone_python_cachedir(self) -> Path:
         return self.home / "py"
 
+    @property
+    def allow_spaces_in_home_path(self) -> bool:
+        return strtobool(os.getenv("PIPX_HOME_ALLOW_SPACE", "0"))
+
     def log_warnings(self):
-        if " " in str(self.home):
+        if " " in str(self.home) and not self.allow_spaces_in_home_path:
             logger.warning(
                 pipx_wrap(
                     (
-                        f"{hazard} Found a space in the home path. We heavily discourage this, due to "
+                        f"{hazard} Found a space in the pipx home path. We heavily discourage this, due to "
                         "multiple incompatibilities. Please check our docs for more information on this, "
                         "as well as some pointers on how to migrate to a different home path."
                     ),

tests/test_environment.py

@@ -2,6 +2,7 @@
 from pathlib import Path
 
 from helpers import run_pipx_cli, skip_if_windows
+from pipx import paths
 from pipx.paths import get_expanded_environ
 
 
@@ -19,6 +20,7 @@ def test_cli(pipx_temp_env, monkeypatch, capsys):
     # Checking just for the sake of completeness
     assert "PIPX_DEFAULT_PYTHON" in captured.out
     assert "USE_EMOJI" in captured.out
+    assert "PIPX_HOME_ALLOW_SPACE" in captured.out
     assert "Environment variables (set by user):" in captured.out
 
 
@@ -33,6 +35,7 @@ def test_cli_with_args(monkeypatch, capsys):
     assert not run_pipx_cli(["environment", "--value", "PIPX_VENV_CACHEDIR"])
     assert not run_pipx_cli(["environment", "--value", "PIPX_DEFAULT_PYTHON"])
     assert not run_pipx_cli(["environment", "--value", "USE_EMOJI"])
+    assert not run_pipx_cli(["environment", "--value", "PIPX_HOME_ALLOW_SPACE"])
 
     assert run_pipx_cli(["environment", "--value", "SSS"])
     captured = capsys.readouterr()
@@ -49,6 +52,29 @@ def test_resolve_user_dir_in_env_paths(monkeypatch):
     assert env_dir is None
 
 
+def test_allow_space_in_pipx_home(
+    monkeypatch,
+    capsys,
+    tmp_path,
+):
+    home_dir = Path(tmp_path) / "path with space"
+    monkeypatch.setattr(paths.ctx, "_base_home", home_dir)
+    assert not run_pipx_cli(["environment", "--value", "PIPX_HOME_ALLOW_SPACE"])
+    paths.ctx.log_warnings()
+    captured = capsys.readouterr()
+    assert "Found a space" in captured.err
+    assert "false" in captured.out
+
+    monkeypatch.setenv("PIPX_HOME_ALLOW_SPACE", "1")
+    assert not run_pipx_cli(["environment", "--value", "PIPX_HOME_ALLOW_SPACE"])
+    paths.ctx.log_warnings()
+    captured = capsys.readouterr()
+    assert "Found a space" not in captured.err
+    assert "true" in captured.out
+
+    paths.ctx.make_local()
+
+
 @skip_if_windows
 def test_cli_global(pipx_temp_env, monkeypatch, capsys):
     assert not run_pipx_cli(["environment", "--global"])
@@ -64,3 +90,4 @@ def test_cli_global(pipx_temp_env, monkeypatch, capsys):
     # Checking just for the sake of completeness
     assert "PIPX_DEFAULT_PYTHON" in captured.out
     assert "USE_EMOJI" in captured.out
+    assert "PIPX_DEFAULT_PYTHON" in captured.out