Packages are not imported from the selected Python environment

[rchiodo]

• Create new conda environment
• Install jupyter into it
• set "jupyter.disableZMQSupport": "true"
• Try running interactive window with this environment

Kernel never starts.

[rchiodo]

Root cause is this when starting the kernel:

  File "C:\Users\aku91\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\jupyter_core\paths.py", line 387, in win32_restrict_file_to_user
    import win32api
ImportError: DLL load failed while importing win32api: The specified procedure could not be found.

It looks like it's half using the global jupyter (which is probably my global jupyter interpreter) and the jupyter installed in the conda environment.

[DonJayamanne]

Works for me, on windows.
@rchiodo Can you run jupyter from within that conda environment?

[rchiodo]

Yes I can. Everything works fine if I start jupyter from that environment. But if I start jupyter in another environment, it doesn't.

[rchiodo]

Figured out the problem. Sys path is wrong.

When working it's this:

['', 'C:\\Users\\Aku91\\Source\\Repos\\PythonApplication1', 'C:\\Users\\aku91\\miniconda3\\envs\\condaTestQT5\\python39.zip', 'C:\\Users\\aku91\\miniconda3\\envs\\condaTestQT5\\DLLs', 'C:\\Users\\aku91\\miniconda3\\envs\\condaTestQT5\\lib', 'C:\\Users\\aku91\\miniconda3\\envs\\condaTestQT5', 'C:\\Users\\aku91\\miniconda3\\envs\\condaTestQT5\\lib\\site-packages', 'C:\\Users\\aku91\\miniconda3\\envs\\condaTestQT5\\lib\\site-packages\\win32', 'C:\\Users\\aku91\\miniconda3\\envs\\condaTestQT5\\lib\\site-packages\\win32\\lib', 'C:\\Users\\aku91\\miniconda3\\envs\\condaTestQT5\\lib\\site-packages\\Pythonwin']

When not working, the global sys path is injected first:

['', 'C:\\Users\\Aku91\\Source\\Repos\\PythonApplication1', 'C:\\Users\\aku91\\miniconda3\\envs\\condaTestQT5\\python39.zip', 'C:\\Users\\aku91\\miniconda3\\envs\\condaTestQT5\\DLLs', 'C:\\Users\\aku91\\miniconda3\\envs\\condaTestQT5\\lib', 'C:\\Users\\aku91\\miniconda3\\envs\\condaTestQT5', 'C:\\Users\\aku91\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python39\\site-packages', 'C:\\Users\\aku91\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python39\\site-packages\\win32', 'C:\\Users\\aku91\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python39\\site-packages\\win32\\lib', 'C:\\Users\\aku91\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python39\\site-packages\\Pythonwin', 'C:\\Users\\aku91\\miniconda3\\envs\\condaTestQT5\\lib\\site-packages', 'C:\\Users\\aku91\\miniconda3\\envs\\condaTestQT5\\lib\\site-packages\\win32', 'C:\\Users\\aku91\\miniconda3\\envs\\condaTestQT5\\lib\\site-packages\\win32\\lib', 'C:\\Users\\aku91\\miniconda3\\envs\\condaTestQT5\\lib\\site-packages\\Pythonwin']

[rchiodo]

I wonder if we can generate a PYTHONPATH that fixes it.

[rchiodo]

Yep that fixes it. @DonJayamanne I fixed it by updating how we print out variables.

I changed this:

import os
import builtins
import json

builtins.print("e976ee50-99ed-4aba-9b6b-9dcd5634d07d")
builtins.print(json.dumps(dict(os.environ)))

to this

import os
import builtins
import json
import sys

builtins.print("e976ee50-99ed-4aba-9b6b-9dcd5634d07d")

# Special case PYTHONPATH to include sys.path
formatted_sys_path = os.pathsep.join(sys.path)
os.environ["PYTHONPATH"] = f"{os.getenv('PYTHONPATH')};{formatted_sys_path}"

# Dump results
builtins.print(json.dumps(dict(os.environ)))

Thoughts? I think this should be okay. We're not removing anything from the PYTHONPATH, and we're ensuring it includes the original sys path for the interpreter.

[DonJayamanne]

Ideally this shouldn't be necessary, also most of the time PYTHONPATH is empty.
I'm concerned we'd be setting that env variables unnecessarily.
Let me have look, ie get a better understanding

Please can you check sys path for other python envs (virtual env) and other conda environments.

[rchiodo]

I'm concerned we'd be setting that env variables unnecessarily.

I believe this should be harmless. PYTHONPATH would be redundant in most cases with what it figures out is sys.path. Although if we got it wrong it would certainly mess up the sys.path.

[rchiodo]

Please can you check sys path for other python envs (virtual env) and other conda environments.

Not sure what this means? You mean when printing? Only do this if you're on a virtual environment?

[DonJayamanne]

1. How did you manage to get the value for sys path (that you've mentioned earlier).
   I'm suggesting we get the same values for another conda env and another virtual env.
2. Concern with PYTHONPATH is, users tend to udpate this in their .env files as well, and we've had issues there too.
   I'm not saying its the wrong approach, just not comfortable without actually figuring out why the path variables are wrong.

[rchiodo]

I'm not saying its the wrong approach, just not comfortable without actually figuring out why the path variables are wrong.

I believe the path variables are wrong because jupyter is injecting its sys.path into the kernel on startup. I got those sys.path variables by changing the kernel launch command.

I switched it from this:

  "argv": [
    "C:\\Users\\aku91\\miniconda3\\envs\\condaTestQT5\\python.exe",
    "-m",
    "ipykernel_launcher",
    "-f",
    "{connection_file}"
  ],

to this

  "argv": [
    "C:\\Users\\aku91\\miniconda3\\envs\\condaTestQT5\\python.exe",
    "-c",
    "import sys;sys.path",
    "-f",
    "{connection_file}"
  ],

It would still crash, but then I could see the sys path.

[rchiodo]

I'm suggesting we get the same values for another conda env and another virtual env.

Not sure what you mean by this? What are they on my machine? Sys.path is what you expect. It includes the site-packages for the venv (or conda env). Conda envs seem to put their own paths first.

[DonJayamanne]

I'm running into similar issues on Windows, this isn't specific to Mac M1 nor is it specific to Jupyter.
I can replicate this in windows even with Raw, the changes made in the PR don't work at all, in fact its a noop.

This is what I'm getting

['c:\\Development\\samples\\pySamples\\sample1\\kernel_issues',
 'C:\\Users\\donja\\anaconda3\\envs\\tf-gpu\\python39.zip',
 'C:\\Users\\donja\\anaconda3\\envs\\tf-gpu\\DLLs',
 'C:\\Users\\donja\\anaconda3\\envs\\tf-gpu\\lib',
 'C:\\Users\\donja\\anaconda3\\envs\\tf-gpu',
 '',
 'C:\\Users\\donja\\AppData\\Roaming\\Python\\Python39\\site-packages',
 'C:\\Users\\donja\\AppData\\Roaming\\Python\\Python39\\site-packages\\win32',
 'C:\\Users\\donja\\AppData\\Roaming\\Python\\Python39\\site-packages\\win32\\lib',
 'C:\\Users\\donja\\AppData\\Roaming\\Python\\Python39\\site-packages\\Pythonwin',
 'C:\\Users\\donja\\anaconda3\\envs\\tf-gpu\\lib\\site-packages',
 'C:\\Users\\donja\\anaconda3\\envs\\tf-gpu\\lib\\site-packages\\win32',
 'C:\\Users\\donja\\anaconda3\\envs\\tf-gpu\\lib\\site-packages\\win32\\lib',
 'C:\\Users\\donja\\anaconda3\\envs\\tf-gpu\\lib\\site-packages\\Pythonwin',
 'C:\\Users\\donja\\.ipython']

Basically the problem is the fact that Python39 is being included into the list of paths (as @rchiodo identified).
& messing things up, when I test the before & after, to me the PR doesn't make any difference.
However, this applies to ALL scenarios (Jupyter as well as Raw, Windows as well as Mac)

The reason for other site_packages being searched (listed in sys.path) is outlined here https://docs.python.org/3/library/site.html#site.ENABLE_USER_SITE
Basically Python adds this by default (i.e. deliberately)
Found that users have complained about this in conda website as well and they just turn this off.

Note:

• IPyKernel from python39 was used to launch tfgpu (which is what @rchiodo came across)
• Packages from conda envionrment where not loaded (i found this by checking the version of packages).

Users could be running into quite a lot of these as well.
Found that Jupyter too has the same problems.

Problems

1. Users don't want pacakges to be imported from anywhere else if they have pacakges installed in their environment.
2. Users want packages to be imported from a global location if its in the global site packages (assumption as this is how the extension has worked all along)
3. When installing packages, users want pakcages to be installed into the environment selected, not global site packages

Solution

1. For problem 1, we turn off site packages, set the ENV variable PYTHONNOUSERSITE
2. For problem 2, we add the default site packages to the bottom of the list in sys.path
3. For problem 3, the solutions for 1 & 2 will be sufficient.

This is what we'd want to resolve 1, 2, & 3

['c:\\Development\\samples\\pySamples\\sample1\\kernel_issues',
 'C:\\Users\\donja\\anaconda3\\envs\\tf-gpu\\python39.zip',
 'C:\\Users\\donja\\anaconda3\\envs\\tf-gpu\\DLLs',
 'C:\\Users\\donja\\anaconda3\\envs\\tf-gpu\\lib',
 'C:\\Users\\donja\\anaconda3\\envs\\tf-gpu',
 '',
 'C:\\Users\\donja\\anaconda3\\envs\\tf-gpu\\lib\\site-packages',
 'C:\\Users\\donja\\anaconda3\\envs\\tf-gpu\\lib\\site-packages\\win32',
 'C:\\Users\\donja\\anaconda3\\envs\\tf-gpu\\lib\\site-packages\\win32\\lib',
 'C:\\Users\\donja\\anaconda3\\envs\\tf-gpu\\lib\\site-packages\\Pythonwin',
 'C:\\Users\\donja\\AppData\\Roaming\\Python\\Python39\\site-packages',
 'C:\\Users\\donja\\.ipython',
 'C:\\Users\\donja\\AppData\\Roaming\\Python\\Python39\\site-packages\\win32',
 'C:\\Users\\donja\\AppData\\Roaming\\Python\\Python39\\site-packages\\win32\\lib',
 'C:\\Users\\donja\\AppData\\Roaming\\Python\\Python39\\site-packages\\Pythonwin']

[DonJayamanne]

I've submitted the PR and added tests for this, should be fine now.
#8576

[DonJayamanne]

Notes

• Re-opening for the team to see this in triage meeting (as the changes were merged without a review)
• Issue was fixed by reverting the previous changes & submitting a new fix
• Please close this issue

[rchiodo]

I verified Don's latest change. It works for me too.

[Brian-Keith]

@DonJayamanne I saw that #8576 as merged and closed, but I'm not sure if it's still working or if something else needs to be changed on my end before the solution works.

I had an issue where I updated plotly in Conda, but it's still showing an old version because it's using the global site packages. Screenshotted below is the output from sys.path and as you can see the conda site packages are prioritized below the global site packages.

Is there something else I need to do? Currently running Python 3.9.12 and VS Code version 1.72.2.