Traceback on Windows 10 involving _psutil_windows

[brettcannon]

Be sure to check the existing issues, both open and closed.

Describe the issue briefly here.

Describe your environment

1. OS Type: Windows 10
2. Python version: $ python -V: 3.6.4
3. Pipenv version: $ pipenv --version: 11.1.2

Expected result

No traceback

Actual result

PS C:\Users\brcan\Desktop\pipenv-test> py -3 -m pipenv install
Creating a virtualenv for this project…
Traceback (most recent call last):
  File "C:\Users\brcan\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Users\brcan\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\brcan\AppData\Local\Programs\Python\Python36\lib\site-packages\pipenv\pew\__main__.py", line 1, in <module>
    import pew
  File "C:\Users\brcan\AppData\Local\Programs\Python\Python36\lib\site-packages\pew\__init__.py", line 3, in <module>
    from . import pew
  File "C:\Users\brcan\AppData\Local\Programs\Python\Python36\lib\site-packages\pew\pew.py", line 39, in <module>
    import psutil
  File "C:\Users\brcan\AppData\Local\Programs\Python\Python36\lib\site-packages\pipenv\vendor\psutil\__init__.py", line 126, in <module>
    from . import _pswindows as _psplatform
  File "C:\Users\brcan\AppData\Local\Programs\Python\Python36\lib\site-packages\pipenv\vendor\psutil\_pswindows.py", line 16, in <module>
    from . import _psutil_windows as cext
ImportError: cannot import name '_psutil_windows'

Virtualenv location:
Creating a Pipfile for this project…
Creating a virtualenv for this project…
Traceback (most recent call last):
  File "C:\Users\brcan\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Users\brcan\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\brcan\AppData\Local\Programs\Python\Python36\lib\site-packages\pipenv\pew\__main__.py", line 1, in <module>
    import pew
  File "C:\Users\brcan\AppData\Local\Programs\Python\Python36\lib\site-packages\pew\__init__.py", line 3, in <module>
    from . import pew
  File "C:\Users\brcan\AppData\Local\Programs\Python\Python36\lib\site-packages\pew\pew.py", line 39, in <module>
    import psutil
  File "C:\Users\brcan\AppData\Local\Programs\Python\Python36\lib\site-packages\pipenv\vendor\psutil\__init__.py", line 126, in <module>
    from . import _pswindows as _psplatform
  File "C:\Users\brcan\AppData\Local\Programs\Python\Python36\lib\site-packages\pipenv\vendor\psutil\_pswindows.py", line 16, in <module>
    from . import _psutil_windows as cext
ImportError: cannot import name '_psutil_windows'

Virtualenv location:
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (625834)!
Installing dependencies from Pipfile.lock (625834)…
  ================================ 0/0 - 00:00:00
To activate this project's virtualenv, run the following:
 $ pipenv shell

Steps to replicate

Command in output above.

[brettcannon]

I also got pipenv trying to use miniconda instead of the Python interpreter I used to launch pipenv when running pipenv shell, but I don't know if that's part of this issue or if it would be a separate one.

[ghost]

Same on 11.1.3

[kennethreitz]

this wasn't happening before?

[ghost]

I reverted to 11.1.0 to resolve my issue temporarily.

[brettcannon]

I have no idea if the issue was occurring previously because I was blocked on #1474 until now.

[kennethreitz]

will get a fix out shortly

[kennethreitz]

once i can reproduce

[randycoulman]

I've been running into this issue today as well. I reverted to pipenv 9.1.0 (since that version is working on another machine) and all is well.

On my systems, psutil gets installed directly in site-packages and has the _psutil_windows.*.pyd file. But in pipenv\vendor\psutil, there is no such *.pyd file. I haven't been able to verify this, but my current speculation is that, in 9.1.0, the site-packages version of psutil is being used, but in 11.1.3, the vendor version is being used and there is no pyd file to load.

[techalchemy]

@kennethreitz @brettcannon I've run into this before, internally we've discussed why we are even vendoring this, it breaks windows on occasion and I don't think anyone knows why it's vendored now.

We might be ok just unvendoring this -- /cc @nateprewitt

FWIW in the past I've resolved this by reinstalling my environment and/or reinstalling pipenv/psutil, no idea why though...

[techalchemy]

/cc @uranusjr for windows knowledge also

[uranusjr]

Pew also depends on psutil on Windows. It was using the site-packages version before, but not it detects the one vendored by Pipenv. My guess is this has something to do with the -m pew change, but I have not idea why it inherits the parent process’s sys.path (while running it with pew.exe does not). This only happens on Windows because pew only imports psutil on Windows.

I’m +1 for unvendoring psutil because this is a bit complicated to solve otherwise.

[uranusjr]

Wondering if this is one of those “subtly different” things. https://twitter.com/dontusethiscode/status/968637241374466053

[techalchemy]

Assuming the PR I just submitted doesn't hit any stumbling blocks building against pypy or nix, I just introduced some changes to just remove psutil from the picture entirely

[uranusjr]

And there you are submitting the “complicated otherwise” solution like it’s nothing… 🤯

[siddhi]

I got this as well on pipenv 11.1.3. It seems like the vendor version of psutil doesnt have the file _psutil_windows.cp36-win_amd64.pyd which is present when I install psutil normally.

[etino]

As @randycoulman I reverto to pipenv 10.1.0 and now all is working fine.

These are the packages version installed on my environment. (Windows 10 64bit with Python 3.6 32bit)

certifi (2018.1.18)
chardet (3.0.4)
emoji (0.4.5)
future (0.16.0)
idna (2.6)
pew (1.1.2)
pip (9.0.1)
pipenv (10.1.0)
psutil (5.3.1)
requests (2.18.4)
setuptools (28.8.0)
urllib3 (1.22)
virtualenv (15.1.0)
virtualenv-clone (0.3.0)

[kennethreitz]

fixed in master

[kennethreitz]

released!

[techalchemy]

@brettcannon @uranusjr @siddhi @joshcurago we are now vendoring a patched version of pew with a native ctypes implementation of the things they were using psutil for, so we have dropped our dependence on psutil and unvendored it, I believe @kennethreitz will be cutting a release shortly

[kennethreitz]

done!

[randycoulman]

I can confirm that 11.1.5 now works for me. Thanks for the quick turnaround!