packaging issues with pipx, usage of sys.executable -m

[AlJohri]

pipx creates virtualenvs by calling out to the terminal using: run([DEFAULT_PYTHON, "-m", "venv", "--clear", self.root]). where DEFAULT_PYTHON is sys.executable.

Is it possible to call out to or expose the embedded python within PyOxidizer?

My top level CLI is pipx but I imagine maybe I can expose the underlying python as pipx python? In which case I can just change my DEFAULT_PYTHON and things can work as is?

I know that it's also possible to create virtualenvs with import venv; venv.create('.venv') but that would be a lot of code to rewrite.

[AlJohri]

I think I should just broaden the question to, is it possible to create usable virtual environments using a PyOxidizer binary?

Using the following configuration, I got pretty close:

[[embedded_python_config]]
raw_allocator = "jemalloc"

[[embedded_python_config]]
sys_paths = ["$ORIGIN/lib"]
ignore_environment = false
filesystem_importer = true

[[packaging_rule]]
type = "stdlib-extensions-policy"
policy = "all"

[[packaging_rule]]
type = "stdlib"
include_source = true
exclude_test_modules = true
install_location = "app-relative:lib"
include_resources = true

[[embedded_python_run]]
mode = "repl"

I can run import venv; venv.create('test-venv', symlinks=True) within the repl.

$ tree test-venv
test-venv
├── bin
│   ├── pipx -> /Users/johria/Development/pipx/build/apps/pipx/x86_64-apple-darwin/debug/pipx
│   ├── python -> pipx
│   └── python3 -> pipx
├── include
├── lib
│   └── python3.7
│       └── site-packages
└── pyvenv.cfg

5 directories, 4 files

However, the resulting symlinked virtual environment cannot be used.

$ ./test-venv/bin/python
Fatal Python error: initfsencoding: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings'

Current thread 0x000000010ec9f5c0 (most recent call first):
zsh: abort      ./test-venv/bin/pipx

Even with trying to provide the environment variables for the python path, it still can't be used:

$ PYTHONPATH="/Users/johria/Development/pipx/build/apps/pipx/x86_64-apple-darwin/debug" ./test-venv/bin/python
Fatal Python error: initfsencoding: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00000001185d45c0 (most recent call first):
zsh: abort      PYTHONPATH= ./test-venv/bin/pipx

[AlJohri]

Linking this issue (#141) which talks about creating a mode to allow for a python-like executable which seems like one of the missing pieces to create a venv using PyOxidizer.

[AlJohri]

Also linking this issue which more explicitly talks about building a python workaline binary: #122

[stereobutter]

@AlJohri I am also very interested in this, have you had a look at this again after #122 has been resolved?