Picking specific VC runtime to use?

[QuLogic]

For Matplotlib 3.3.1, we started using cibuildwheel to produce wheels in GitHub Actions. Previously, they were built by Christoph Gohlke and then posted by us.

We got a report on 3.3.1 that the runtime requirements changed from VCRUNTIME140.dll to VCRUNTIME140_1.dll. In the cibuildwheel docs, I see something about Python 2 for Windows, but nothing specific about VC runtimes. Note, as we do not build for Python 2, I did not install Visual C++ for Python 2.7.

Is there some way we can configure which runtime gets linked?

[Czaki]

cibuildwheel does not manage windows compliers. And there is no auditwheel/delocate alternative.
It could be somehow controlled by environment variables. First think to check is if github actions hosted worker contains VCRUNTIME140.dll file. It yes then I thinks something similar to python 2.7 changes should work.

Unfortunately github actions contains only one windows image. Other possible workaround is to check if Azure Pipelines vs2017-win2016 runner build contains reference to proper version of VCRUNTIME140 https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops&tabs=yaml

[henryiii]

I think the problem is here:

https://developercommunity.visualstudio.com/content/problem/852548/vcruntime140-1dll-is-missing.html

Starting in 2019, the feature that requires VCRUNTIME140_1.dll is now on by default. Either downgrading to the 2017 image (not ideal), or "disabling FH4 by passing -d2FH4- compiler flag and -d2:-FH4- linker flag" ought to fix it. Most redistributable packages now have it, but older installs will be missing it.

@YannickJadoul is looking into it.

[YannickJadoul]

Seems @henryiii is right!

I was playing around with some (CMake) builds and depends.exe and found the following:

• By default, VS 2019 seems to link against both VCRUNTIME140.dll and VCRUNTIME140_1.dll.
• By default, VS 2017 seems to link against only VCRUNTIME140.dll. (Maybe this is what Christoph Gohlke is using to build those wheels? Would be nice to double check, just for sanity.)
• Adding -DCMAKE_CXX_FLAGS=/d2FH4- to my CMake configuration invocation seems to remove the VCRUNTIME140_1.dll dependency. I guess this can just as well be added to setup.py. Worth a try, I'd say?

[YannickJadoul]

Apart from that, @Czaki is right, as well, btw. We don't control the Visual Studio installation or MSVC invocation in cibuildwheel, but depend on the CI's environment for this. (@Czaki's also right about using VS 2017, but using that might become harder and harder in the future?)
I guess we could add something to cibuildwheel to automatically add that, but I'm very hesitant to do so. Maybe it's better to just add a note to the "un(clearly )documented MSVC/VS quirks" part of the docs?

[Czaki]

Maybe it's better to just add a note to the "un(clearly )documented MSVC/VS quirks" part of the docs?

Vote for this.

[henryiii]

One other thing; GitHub and Azure use the same runners, windows-2016 is the GitHub Actions one; it isn't really publicized and might disappear someday. A solution without it would be better long-term, but it is there for now.

I think this is just documentation on cibuildwheel's side - the problem with matplotlib was the upgrade from 2017 to 2019, not cibuildwheel; but it's something that would be best noted with the potential workaround.

[YannickJadoul]

One other thing; GitHub and Azure use the same runners, windows-2016 is the GitHub Actions one; it isn't really publicized and might disappear someday. A solution without it would be better long-term, but it is there for now.

Interesting. I knew they were similar, but not that you could use the same images in both!

OK, I'll write something up, once @QuLogic would find some time to check out the proposed solutions and confirm that they also work in matplotlib's case!

[Czaki]

Both GitHub and Azure are owned by Microsoft. So why double job.

[QuLogic]

I was able to figure out how to pass that flag through to our extensions here: matplotlib/matplotlib#18322 and afaict is working to reduce the runtime requirements.

[YannickJadoul]

Great!

So, TODO: write a quick note, somewhere in the docs :-) I should be able to find 15 minutes somewhere this week.

[YannickJadoul]

I've created #424. Mind having a look and adding your thoughts/remarks, @QuLogic? Does this explain the situation from this issue well enough?