PyQt -- ImportError: DLL load failed: The specified module could not be found

[tjguk]

Attempting to start Mu in a virtualenv on Windows gives an ImportError. NB this does not happen with a stdlib venv.

From within a fresh clone on Windows:

• py -3 -mvirtualenv .venv
• .venv\scripts\activate
• python -mmu

Traceback (most recent call last):
  File "C:\Program Files\Python36\Lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Program Files\Python36\Lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "c:\work-in-progress\mu\mu\__main__.py", line 1, in <module>
    from mu.app import run
  File "c:\work-in-progress\mu\mu\app.py", line 46, in <module>
    from PyQt5.QtCore import QTimer
ImportError: DLL load failed: The specified module could not be found.

This is because QtCore (and other Qt extensions) are linked to python3.dll rather than to python36.dll. The released versions of virtualenv (15.1 as I write) do not copy this file over. virtualenv master has a fix for this (cf pypa/virtualenv#1083) but no released version has this fix.

The stdlib venv module does copy this file, so the advice is: use python -mvenv on Windows

[ntoll]

Just to confirm my reading of this...

• This problem will eventually go away (there is a fix in the works).
• In any case, we should just use python -mvenv in our docs.

[carlosperate]

I had the same problem in #300, however since in that machine I was using a version of virtualenv that was probably a couple of years old, I wrongly assumed it was an old bug likely fixed already.

As I like to use virtualenvwrapper what I ended up doing was manually copy the python3.dll file to the virtualenv directory. It's very hackish, but does the job until this is fixed properly.

We should add a note to the docs and keep this issue opened until there is a new release with the fix.

[tjguk]

@ntoll yes to both. I think that people (like @carlosperate ) who choose to use one of the venv wrappers are assumed to know what they're about. It might be worth an FAQ to say that "to fix [the ImportError] copy the python3.dll" (which is what I did as well!)

[ntoll]

Aha... understood.

Since I've never encountered this error, I'm not quite sure what exactly I should be writing and where. It would be really helpful if someone could add the right thing to the right place in the docs (sorry).

[tjguk]

I'll add something to the docs -- when I work out where!

[ntoll]

In here perhaps..? https://github.com/mu-editor/mu/blob/master/docs/setup.rst

[carlosperate]

I'm getting this same error on Windows 7 when using the windows installer:

C:\Users\IEUser>C:\Users\IEUser\AppData\Local\Mu\Python\python.exe "C:\Users\IEUser\AppData\Local\Mu\Mu.launch.pyw"
Traceback (most recent call last):
  File "C:\Users\IEUser\AppData\Local\Mu\Mu.launch.pyw", line 30, in <module>
    from mu.app import run
  File "C:\Users\IEUser\AppData\Local\Mu\pkgs\mu\app.py", line 46, in <module>
    from PyQt5.QtCore import QTimer
ImportError: DLL load failed: The specified module could not be found.

To reproduce:

• Install virtualbox
• Download free Windows VM for testing IE, specifically the "IE8 on Windows 7 (x86)"
• Open the Windows VM
• Download and install mu_2018-03-04_12_28_master_8236b4b_32bit.exe
• Trying to open Mu fails silently (app doesn't open, nothing happens)
• Find the Mu shortcut, copy the command and paste into a command prompt changing pythonw.exe to python.exe.
  So, if installed in default location for a single user, that would be: C:\Users\IEUser\AppData\Local\Mu\Python\python.exe "C:\Users\IEUser\AppData\Local\Mu\Mu.launch.pyw"

Looking at the included files, both python3.dll and python36.dll are there:

C:\Users\IEUser\AppData\Local\Mu\Python>tree /F /A
Folder PATH listing for volume Windows 7
Volume serial number is 00000200 D055:099C
C:.
    api-ms-win-core-console-l1-1-0.dll
    api-ms-win-core-datetime-l1-1-0.dll
    api-ms-win-core-debug-l1-1-0.dll
    api-ms-win-core-errorhandling-l1-1-0.dll
    api-ms-win-core-file-l1-1-0.dll
    api-ms-win-core-file-l1-2-0.dll
    api-ms-win-core-file-l2-1-0.dll
    api-ms-win-core-handle-l1-1-0.dll
    api-ms-win-core-heap-l1-1-0.dll
    api-ms-win-core-interlocked-l1-1-0.dll
    api-ms-win-core-libraryloader-l1-1-0.dll
    api-ms-win-core-localization-l1-2-0.dll
    api-ms-win-core-memory-l1-1-0.dll
    api-ms-win-core-namedpipe-l1-1-0.dll
    api-ms-win-core-processenvironment-l1-1-0.dll
    api-ms-win-core-processthreads-l1-1-0.dll
    api-ms-win-core-processthreads-l1-1-1.dll
    api-ms-win-core-profile-l1-1-0.dll
    api-ms-win-core-rtlsupport-l1-1-0.dll
    api-ms-win-core-string-l1-1-0.dll
    api-ms-win-core-synch-l1-1-0.dll
    api-ms-win-core-synch-l1-2-0.dll
    api-ms-win-core-sysinfo-l1-1-0.dll
    api-ms-win-core-timezone-l1-1-0.dll
    api-ms-win-core-util-l1-1-0.dll
    api-ms-win-crt-conio-l1-1-0.dll
    api-ms-win-crt-convert-l1-1-0.dll
    api-ms-win-crt-environment-l1-1-0.dll
    api-ms-win-crt-filesystem-l1-1-0.dll
    api-ms-win-crt-heap-l1-1-0.dll
    api-ms-win-crt-locale-l1-1-0.dll
    api-ms-win-crt-math-l1-1-0.dll
    api-ms-win-crt-multibyte-l1-1-0.dll
    api-ms-win-crt-private-l1-1-0.dll
    api-ms-win-crt-process-l1-1-0.dll
    api-ms-win-crt-runtime-l1-1-0.dll
    api-ms-win-crt-stdio-l1-1-0.dll
    api-ms-win-crt-string-l1-1-0.dll
    api-ms-win-crt-time-l1-1-0.dll
    api-ms-win-crt-utility-l1-1-0.dll
    pyexpat.pyd
    python.exe
    python3.dll
    python36.dll
    python36.zip
    python36._pth
    pythonw.exe
    select.pyd
    sqlite3.dll
    ucrtbase.dll
    unicodedata.pyd
    vcruntime140.dll
    winsound.pyd
    x86.zip
    _asyncio.pyd
    _bz2.pyd
    _ctypes.pyd
    _decimal.pyd
    _elementtree.pyd
    _hashlib.pyd
    _lzma.pyd
    _msi.pyd
    _multiprocessing.pyd
    _overlapped.pyd
    _socket.pyd
    _sqlite3.pyd
    _ssl.pyd

No subfolders exist

[tjguk]

Can you run under procmon? Should be able to spot which DLL is being sought.

[tjguk]

BTW I note you're running a 32-bit install. Is it possible we've captured the wrongly-bitted DLLs? (Just a thought)

[ntoll]

It was working with the installer... I'll take a look.

[carlosperate]

Here is the procmon log:
Logfile.PML.zip
Interesting ones:

[ntoll]

Huh... so that's a list of NAME NOT FOUND and PATH NOT FOUND errors. Do those files exist?

I notice you appear to have used the installer to install Mu just for you, rather than for all users (i.e. in "Program Files"). Could this be a source of problems..?

Also, what happens if you uninstall and reinstall it..? (Just curious)

[tjguk]

I don't have time to pick up the log right now, but the way Python's import mechanism works generates a lot of NAME/PATH NOT FOUND as it scans sys.path and then the system PATH looking for modules and .DLLs. So lots of NOT FOUND is somewhat expected.

[ntoll]

Ahh, good to know. Thanks @tjguk for explaining. This is very much in the Twilight Zone of my Windows knowledge. ;-)

[carlosperate]

I notice you appear to have used the installer to install Mu just for you, rather than for all users (i.e. in "Program Files"). Could this be a source of problems..?

Yes, my first instict was to change it to all users, but I though I should try the default option, which is just for the current user. This is also likely the option that requires no lower admin rights to install.

Uninstalled and reinstalled: Same result
Uninstaller then reinstaled for all users: Same result

[ntoll]

@carlosperate thank you for going through this pain, and so sorry to have inflicted it upon you! :-/

You appear to be using the same sort of VM to me when I was checking the installer was working. I wonder what's different. I'll try to take a look this afternoon.

[carlosperate]

No worries, I was just trying to replicate #373 and trying in Windows 7 in case there was also CRT dll issues. While the two issues might not be caused by the same problem it's probably a matter of figuring out missing dlls and both could be debugged and fixed at the same time.

[ntoll]

I just remembered that I recently bumped the version of Qt up a minor "notch". I wonder if that may be causing the problem. I'll revert it back to what it was and re-test this afternoon (once I've finished what I'm doing this morning!). ;-)

[tjguk]

On the surface, looks like it's this:

https://stackoverflow.com/questions/33265663/api-ms-win-crt-runtime-l1-1-0-dll-is-missing-when-opening-microsoft-office-file

@carlosperate can you check whether the KB update mentioned there is applied to your box?

[ntoll]

OK... so I found several gremlins in the Windows packaging where there was a mismatch between the different versions of Qt related things. I've fixed it all up and tested both 32bit (on Windows 7) and 64bit (on Windows 10) and they work for me (tm). ;-)

Please try the latest builds of master and let me know.

[carlosperate]

Tried https://s3-eu-west-2.amazonaws.com/mu-builds/windows/mu_2018-03-08_13_52_master_981c909_32bit.exe (latest build at the time or writing) and I have the same problem:

C:\Users\IEUser>"C:\Program Files\Mu\Python\python.exe" "C:\Program Files\Mu\Mu.launch.pyw"
Traceback (most recent call last):
  File "C:\Program Files\Mu\Mu.launch.pyw", line 30, in <module>
    from mu.app import run
  File "C:\Program Files\Mu\pkgs\mu\app.py", line 46, in <module>
    from PyQt5.QtCore import QTimer
ImportError: DLL load failed: The specified module could not be found.

[ntoll]

hmm... so what's different about your Windows and mine (Windows 7, 32bit) while installing exactly the same package. :-/

[ntoll]

Forgot to say, I just tried the package you reference with no issues whatsoever. :-/ sigh computers are frustrating...

[carlosperate]

Probably the Windows CRT (it's like the VC++ redistributables for applications built against the newest SDK) as suggested by @tjguk. This was also the main reason we had to stick with Python 3.4 (and the last PyQt for Py3.4) when using PyInstaller.
There are a lot of CRT dlls already included the embedded python, maybe there is something missing.

[ntoll]

As far as I can tell, if you have an up-to-date Windows, everything works. I've tested on Windows 7 and 10. I think it reasonable that our users keep their OS up-to-date.

[carlosperate]

This is not categorised as "important" update, but an "Optional/Recommended" update, so we cannot count on all Windows 7/8/8.1 users with an up-to-date OS to have this installed. However, I will agree that the % of users in this category will be relatively small.

If this Universal CRT is the problem, the solution would be to ensure all required DLLs are packed with Mu. A lot of these are already included, as shown in #354 (comment), so maybe it's just a matter of finding out what might be missing.

I will also point out that we haven't really confirmed yet if this issue is indeed caused by the Windows CRT. The Microsoft provided Windows 7 virtualbox image is not an unresonable windows configuration, so we do have an issue. However, we can cross that brigde when we come to it, we might not have any users affected.

[ntoll]

Well, in the (upcoming)new website, the instructions will say that we expect users to have up-to-date systems including recommended updates. Quite understand that it's not confirmed caused by Windows CRT, but, try as I might, I can't reproduce this problem with the available Windows I have (which all happen to be up to date).

[ntoll]

It's good to see WIndows robustly working for me, no matter how hard I try to make it break. :-P

[carlosperate]

@ntoll We got a report about Mu crashing on startup on several of their computers and found the error to be this one.

...
        from PyQt5.QtCore import QTimer
ImportError: DLL load failed: The specified module could not be found.

It was fixed by installing the Windows Universal CRT package from https://support.microsoft.com/en-us/help/2999226/update-for-universal-c-runtime-in-windows.

However, this problem can be specially unfriendly as it presents itself as the application refusing to launch and there aren't any errors shown.
We were able to capture the trace it by running the bundled Python interpreter from the command line, which can be difficult to request remotely for inexperienced users.
In this specific case their original workaround was simply to install/copy an older version of Mu.

There is a small line in the website about using an up-to-date version of Windows:

It’s a good idea to make sure your system to up-to-date with all Windows updates.

But to be completely honest, and this is just an opinion without any empirical data, I would guess most people feel comfortable enough running a Windows installer, that they are unlikely to carefully read these instructions, and much less to extrapolate from this that they need to update Windows, or enable the recommended updates.

If you install something in a in-working-order computer and Mu is the only application that fails to launch the first assumption for most people would probably have is that problem is within Mu and not the OS.

We should probably add more details about this required Windows update in the installation instructions, but I do wonder if that's enough, as I'm not sure how visible it is there.
It would be more visible in the downloads page, but I do also wonder if a note there for such a small number of users is worth the additional "visual noise". Thoughts?

Apart from all this, looking at pynsist it seems like it is already bundling some of these files and that it copies the DLLs along the app when necessary (this is for Python itself to run):
https://github.com/takluyver/pynsist/blob/eabbdca6de3776ff9d098bd769182d11ab23d002/nsist/pyapp_msvcrt.nsi#L6
Perhaps this is something that could also be implemented within pynsist? And in that case it would work in Windows installation without this recommended update.