R: Move to UCRT on Windows

[xhochy]

CRAN is moving to use URCT as the C library next month. This will potentially lead to interoperability issues between R packages from conda-forge and ones from CRAN. This leads to the following questions/potential issues:

1. Can we update our mingw-toolchain to support UCRT based builds?
2. Do we need to introduce a mutex to not mix old and new builds?
3. Will this have an impact on the interoperability between the non-R and R packages?

Useful background information:

• Adding UCRT to the build matrix msys2/MINGW-packages#6901
• https://developer.r-project.org/Blog/public/2020/05/02/utf-8-support-on-windows/index.html
• https://mingwpy.github.io/ucrt.html
• https://www.msys2.org/docs/environments/

Things to do

• Get a list of all packages that we need. (Search for m2-*, m2w64-*, {{native}}*, {{posix}}* in all the feedstocks after cloning https://github.com/conda-forge/feedstocks/tree/main/feedstocks
• Get a snapshot of all the files in https://repo.msys2.org/msys/x86_64/ and https://repo.msys2.org/msys/sources/
  • Put it in a release tag of https://github.com/conda-forge/msys2-recipes
  • Maybe not all of them. Just the ones we need from the previous step.
• Compiler activation packages
• tktable for win-64

[jakirkham]

cc @conda-forge/r

[isuruf]

Can we update our mingw-toolchain to support UCRT based builds?

Yes, but will need a lot of work.

Do we need to introduce a mutex to not mix old and new builds?

Yes, there is already an epoch which is used by all m2w64-* packages and we can update the epoch. Downstream packages have the epoch in run, but we should add it to downstream packages. It's important to not mix.

Will this have an impact on the interoperability between the non-R and R packages?

No, it will make the interoperability better. (CRT objects can be allocated and deallocated across DLL boundaries with MSVC and MinGW ucrt build)

[jeroen]

Hello, upstream maintainer of many R packages and former R toolchain maintainer here :)

The safest and easiest solution would be to start using the toolchain from the rtools42 bundle provided by CRAN:

• docs: https://cran.r-project.org/bin/windows/Rtools/rtools42/rtools.html
• tarball: https://cran.r-project.org/bin/windows/Rtools/rtools42/files/rtools42-toolchain-libs-base-5111.tar.zst

This contains the mingw-w64 gcc-10.3 toolchain that has been tested to work with all CRAN packages, and also includes all 3rd party system libs as assumed by CRAN packages.

Using custom compilers will become increasingly difficult, because R packages either have to hardcode the linker flags for external libraries as they work with rtools42, or alternatively download their own static ucrt external libraries at install time. Trying to make those packages work with your custom toolchains will probably be an ongoing maintenance nightmare :)

[jakirkham]

xref: #1044

[dpryan79]

Given that this is presumably limiting the R 4.2.0 builds should windows builds "simply" be skipped for R packages for now?

[scdub]

I realize this is a massive undertaking and will require non-trivial resources being drawn away from other important work. Is there a place to discuss this effort and when it might make sense for conda-forge? I'm aware of a few workflows which are broken on 4.1 but do work on 4.2 on Windows because of the change in toolchains. Thanks for everything you all do.

[h-vetinari]

The safest and easiest solution would be to start using the toolchain from the rtools42 bundle provided by CRAN

Some additional context from conda-forge/r-base-feedstock#210 for those arriving on this issue:

Is there a reason not to use rtools as jeroen suggested [...]?

For two reasons,

1. We want the builds to be compatible with the rest of the conda ecosystem.
2. rtools uses static builds which we do not want to do.

[jakirkham]

Given this migration is now underway with R 4.4 in conda-forge ( conda-forge/conda-forge-pinning-feedstock#6040 ), should we close this out?

Admittedly the migration is not done, but it is moving at a good clip. Efforts are probably best directed to the various r-*-feedstocks