-
-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
3.11 support #2440
Comments
PR welcome. |
I'm not totally sure virtualenv is doing it wrong seeing as how this has been consistent for previous 3.x versions up until 3.11 and the reworking of I'm not well versed enough in virtualenv to make real suggestions,.. I assume we're intentionally searching for the executable one level higher than current? |
@gaborbernat One option i was considering was tweaking diff --git a/src/virtualenv/discovery/py_info.py b/src/virtualenv/discovery/py_info.py
index 7c680ee..755773c 100644
--- a/src/virtualenv/discovery/py_info.py
+++ b/src/virtualenv/discovery/py_info.py
@@ -138,6 +138,19 @@ class PythonInfo(object):
base_executable = getattr(sys, "_base_executable", None) # some platforms may set this to help us
if base_executable is not None: # use the saved system executable if present
if sys.executable != base_executable: # we know we're in a virtual environment, cannot be us
+ # Do light validation for non-Windows virtual environments since some installs/distributions
+ # do not provide a version-less "python" binary, so try to fallback to an alternative
+ # This should still be relatively fast since it's just a couple of `stat` calls
+ if self.os != "nt" and (self.version_info.major, self.version_info.minor) >= (3, 11):
+ if not os.path.exists(base_executable):
+ major, minor = self.version_info.major, self.version_info.minor
+ base_dir = os.path.dirname(base_executable)
+ for candidate in [
+ os.path.join(base_dir, f"python{v}") for v in (major, f"{major}.{minor}")
+ ]:
+ if os.path.exists(candidate):
+ base_executable = candidate
+ break
return base_executable This essentially mimics the logic i put in my PR to cpython to fix this issue, but the chances of that getting accepted seems low and if it was, the timeline is unknown. Even if it is, there's a gap for 3.11.0 where a non-existent binary will be returned from running a couple of Does this seem like a reasonable interim fix until there's something in cpython? |
Can you show what the diff would be to the current state? |
I've updated the snippet to be a diff In terms of results, here's what it would look like: installed python from sources:
make a venv:
before the fix to py_info:
Note that "python" is not available in the system installation. after fix:
The result now maps to a legitimate binary available in the directory pointed at by "home" from For situations where "home" is incorrect (fixed in my PR), there will be no change in behavior, there are just 3 extra |
Fill in the PR seems reasonable, with some small changes I can comment on the PR. |
|
Upstream is considering my PR for the fix in main. I have no idea if it will be backported to 3.11 at this point. |
@gaborbernat thanks! is there a way to get releases pushed to bootstrap? https://bootstrap.pypa.io/virtualenv/ |
I forgot to do that. Will do later today.
Thanks for the reminder.
…On Sat, Nov 12, 2022, 16:04 Vincent Fazio ***@***.***> wrote:
@gaborbernat <https://github.com/gaborbernat> thanks!
is there a way to get releases pushed to bootstrap?
https://bootstrap.pypa.io/virtualenv/
—
Reply to this email directly, view it on GitHub
<#2440 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAFIQPWQBHG66NLS2TQ2DR3WIAWBPANCNFSM6AAAAAARXSVWNU>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Done. |
It says 20.16.6 😭 |
Not anymore pypa/get-virtualenv@d3b67d1 ... |
Bumps [virtualenv](https://github.com/pypa/virtualenv) from 20.16.6 to 20.16.7. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/pypa/virtualenv/releases">virtualenv's releases</a>.</em></p> <blockquote> <h2>20.16.7</h2> <h2>What's Changed</h2> <ul> <li>release 20.16.6 by <a href="https://github.com/gaborbernat"><code>@gaborbernat</code></a> in <a href="https://github-redirect.dependabot.com/pypa/virtualenv/pull/2437">pypa/virtualenv#2437</a></li> <li>Replace six in tests/unit/test_run.py by <a href="https://github.com/cjmayo"><code>@cjmayo</code></a> in <a href="https://github-redirect.dependabot.com/pypa/virtualenv/pull/2439">pypa/virtualenv#2439</a></li> <li>Try to fix Nushell install by <a href="https://github.com/kubouch"><code>@kubouch</code></a> in <a href="https://github-redirect.dependabot.com/pypa/virtualenv/pull/2444">pypa/virtualenv#2444</a></li> <li>Try alternate filenames for system_executable by <a href="https://github.com/vfazio"><code>@vfazio</code></a> in <a href="https://github-redirect.dependabot.com/pypa/virtualenv/pull/2442">pypa/virtualenv#2442</a></li> <li>Bump embedded by <a href="https://github.com/gaborbernat"><code>@gaborbernat</code></a> in <a href="https://github-redirect.dependabot.com/pypa/virtualenv/pull/2443">pypa/virtualenv#2443</a></li> <li>Set 'home' to parent directory of system_executable by <a href="https://github.com/vfazio"><code>@vfazio</code></a> in <a href="https://github-redirect.dependabot.com/pypa/virtualenv/pull/2441">pypa/virtualenv#2441</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/cjmayo"><code>@cjmayo</code></a> made their first contribution in <a href="https://github-redirect.dependabot.com/pypa/virtualenv/pull/2439">pypa/virtualenv#2439</a></li> <li><a href="https://github.com/vfazio"><code>@vfazio</code></a> made their first contribution in <a href="https://github-redirect.dependabot.com/pypa/virtualenv/pull/2442">pypa/virtualenv#2442</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/pypa/virtualenv/compare/20.16.6...20.16.7">https://github.com/pypa/virtualenv/compare/20.16.6...20.16.7</a></p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pypa/virtualenv/blob/main/docs/changelog.rst">virtualenv's changelog</a>.</em></p> <blockquote> <h2>v20.16.7 (2022-11-12)</h2> <p>Bugfixes - 20.16.7</p> <pre><code>- Use parent directory of python executable for pyvenv.cfg "home" value per PEP 405 - by :user:`vfazio`. (`[#2440](pypa/virtualenv#2440) <https://github.com/pypa/virtualenv/issues/2440>`_) - In POSIX virtual environments, try alternate binary names if ``sys._base_executable`` does not exist - by :user:`vfazio`. (`[#2442](pypa/virtualenv#2442) <https://github.com/pypa/virtualenv/issues/2442>`_) - Upgrade embedded wheel to ``0.38.4`` and pip to ``22.3.1`` from ``22.3`` and setuptools to ``65.5.1`` from ``65.5.0`` - by :user:`gaborbernat`. (`[#2443](pypa/virtualenv#2443) <https://github.com/pypa/virtualenv/issues/2443>`_) </code></pre> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pypa/virtualenv/commit/d536b25fd38699a15cb9d782be2786d2aec4de83"><code>d536b25</code></a> release 20.16.7</li> <li><a href="https://github.com/pypa/virtualenv/commit/cf36e4f485fe06c9b52dd657c02b84867e4b8bce"><code>cf36e4f</code></a> Set 'home' to parent directory of system_executable (<a href="https://github-redirect.dependabot.com/pypa/virtualenv/issues/2441">#2441</a>)</li> <li><a href="https://github.com/pypa/virtualenv/commit/b7178cdbfe1e6b59734137c142ae260673698a0e"><code>b7178cd</code></a> Bump embedded (<a href="https://github-redirect.dependabot.com/pypa/virtualenv/issues/2443">#2443</a>)</li> <li><a href="https://github.com/pypa/virtualenv/commit/50b4b6b72a0709501b36f714e317b07c3d765833"><code>50b4b6b</code></a> Try alternate filenames for system_executable (<a href="https://github-redirect.dependabot.com/pypa/virtualenv/issues/2442">#2442</a>)</li> <li><a href="https://github.com/pypa/virtualenv/commit/dbc57c2bd6c9d23d482bc7400ddb6c7c7022de09"><code>dbc57c2</code></a> Try to fix Nushell install (<a href="https://github-redirect.dependabot.com/pypa/virtualenv/issues/2444">#2444</a>)</li> <li><a href="https://github.com/pypa/virtualenv/commit/e4a42c6eeb48004461bb8711fa06e830bc40eb37"><code>e4a42c6</code></a> Fix nushell install</li> <li><a href="https://github.com/pypa/virtualenv/commit/c13db75469d6d39778049930431f3496c1e2c4ac"><code>c13db75</code></a> Fix documentation build</li> <li><a href="https://github.com/pypa/virtualenv/commit/2224abb367286cd2d6b888b0e22ee4d108ec6b74"><code>2224abb</code></a> Replace six in tests/unit/test_run.py (<a href="https://github-redirect.dependabot.com/pypa/virtualenv/issues/2439">#2439</a>)</li> <li><a href="https://github.com/pypa/virtualenv/commit/1e90b97f70f03b1fc6bde3b187bdc9d9f4590f7e"><code>1e90b97</code></a> Simplify docs conf</li> <li><a href="https://github.com/pypa/virtualenv/commit/ba6ac17c9b058aadcddd95e2a3ec689a2c3c9b69"><code>ba6ac17</code></a> Bump tools and deps</li> <li>Additional commits viewable in <a href="https://github.com/pypa/virtualenv/compare/20.16.6...20.16.7">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=virtualenv&package-manager=pip&previous-version=20.16.6&new-version=20.16.7)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details>
Issue
On python 3.11+, virtualenv returns an invalid
system_executable
for python binaries out of virtual environments made with--copies
See:
python-poetry/poetry#6940 (comment)
Part of this is due to a change in 3.11 that changes the value of
sys._base_executable
to substitute the value of the "home" key from pyvenv.cfg.https://bugs.python.org/issue46028
python/cpython#29041
python/cpython#30144
On it's own, Virtualenv is setting this to a path that does not actually include the python binary so the system_executable is "wrong"
For virtual environments created from a venv parent, subsequent virtualenvs continue to return a binary that does not exist. Poetry is a good example:
This causes virtualenv to fail to create virtual environments due to not being able to find the system executable
Stack:
I've proposed a fix in upstream cpython to try to paper over this discrepancy between binaries available in a venv and what's available in the system install path. I don't expect that to move quickly. Virtualenv could, in the interim, catch this known problematic scenario and return a system python that does exist.
The text was updated successfully, but these errors were encountered: