From 5b92653caebce2942be37e95236f49aa346ced76 Mon Sep 17 00:00:00 2001 From: "H. Vetinari" Date: Sat, 22 Apr 2023 10:28:36 +1100 Subject: [PATCH] update infrastructure.rst/compilers --- src/maintainer/infrastructure.rst | 98 ++++++++++++++++++++++--------- 1 file changed, 69 insertions(+), 29 deletions(-) diff --git a/src/maintainer/infrastructure.rst b/src/maintainer/infrastructure.rst index ce58ec6c85..e059598c42 100644 --- a/src/maintainer/infrastructure.rst +++ b/src/maintainer/infrastructure.rst @@ -281,35 +281,75 @@ Compilers and Runtimes Conda-forge builds and maintains its own set of compilers for various languages and/or systems (e.g., ``C``, ``FORTRAN``, ``C++``, ``CUDA``, etc.). These are used -in all of our CI builds to build both core dependencies (e.g., ``Python``) and maintainer-contributed -packages. While we do not have any formal policies or promises of support for these -compilers, we have historically maintained them according to the following (non-binding) -principles. - -* The authoritative source of the current compilers and versions for various languages - and platforms is the `conda_build_config.yaml `_ - in the `conda-forge/conda-forge-pinning-feedstock `_ - as described in :ref:`globally_pinned_packages`. -* We provide no support of any kind in terms of the long-term stability of these pinnings. -* We upgrade them in an ad-hoc manner on a periodic basis as we have the time and energy to do so. - Note that because of the way we enforce runtime constraints, these compiler upgrades will not break - existing packages. However, if you are using the compilers outside of ``conda``, then you may find issues. -* We generally provide notice in the form of an announcement when a compiler is going to be upgraded. - Note that these changes take a bit of time to complete, so you will generally have time - to prepare should you need to. -* Some of the criteria we think about when considering a compiler migration include - 1) the degree of disruption to the ecosystem, 2) the amount of work for the ``core`` team, - and 3) the amount of time it will cost our (volunteer) feedstock maintainers. - -We do use some unofficial names for our compiler stack internally. Note however that -the existence of these names does not imply any level of support or stability for the compilers -that form the given stack. - -* Our current compiler stack is referred to internally as ``comp7``. -* The previous compiler stack based in part on the various ``toolchain_*`` packages - was sometimes referred to as ``comp4``. On linux the ``toolchain_*`` compilers were - GCC 4.8.2 as packaged in the devtoolset-2 software collection. On osx, we use clang from - Apple's Xcode in the ``toolchain_*`` packages. +in all of our CI builds to build essentially all artefacts published by conda-forge. + +In general, our compilers on Linux and OSX are using very recent compilers, whereas +on windows, we generally use the last supported VS version. +Note that we do not have any formal policies or promises of support for these +compilers, they are primarily intended for use within conda-forge CI. Despite this, +we try to keep them working and even provide an easy way to install them (through +the `compilers`_ feedstock). + +The currently used compilers and versions for various languages and platforms +can be found in the `conda_build_config.yaml `_ +of the `conda-forge/conda-forge-pinning-feedstock `_ +as described in :ref:`globally_pinned_packages`. + +In more detail, our default compiler stack is made up very differently on each platform. +More specifically, each compiler needs an _activation_ that makes the difference +between it being merely present in a build environment, and it being used by default. +These will be installed when using `{{ compiler('xyz') }}` in `meta.yaml`, where `'xyz'` +is one of `'c', 'cxx', 'fortran', 'cuda', 'rust', 'go-cgo', 'go-nocgo'`. + +Linux: +* [C, C++, Fortran] Activation: https://github.com/conda-forge/ctng-compiler-activation-feedstock/ +* [C, C++, Fortran] Implementation: https://github.com/conda-forge/ctng-compilers-feedstock +* Note that when used in conjunction with CUDA, compiler versions are restricted by the + maximum GCC version supported by nvcc (which is also reflected in the global pinning). + +OSX: +* [C, C++] Activation: https://github.com/conda-forge/clang-compiler-activation-feedstock/ +* [C, C++] Required feedstocks: + `llvmdev`_, + `clangdev`_, + `compiler-rt`_, + `libcxx`_, + `openmp`_, + `lld`_, + `cctools`_ +* [Fortran] Activation: https://github.com/conda-forge/gfortran_osx-64-feedstock/ +* [Fortran] Implementation: https://github.com/conda-forge/gfortran_impl_osx-64-feedstock/ + +Windows: +* [C, C++] Activation: https://github.com/conda-forge/vc-feedstock +* [C, C++] We cannot redistribute the actual MSVC compilers due to licensing constraints +* [Fortran] Activation & Implementation: https://github.com/conda-forge/flang-feedstock + +There exists an alternative, MinGW-based, compiler stack on windows, which is available +with a `m2w64_` prefix (e.g. `{{ compiler('m2w64_c') }}`), which is however falling out +of use now that most projects will natively support compilation also with MSVC, in addition +to several complications arising from mixing compiler stacks. + +Additionally, there is a possibility to use clang as a compiler also on Linux & Windows: +* Activation (Linux): https://github.com/conda-forge/clang-compiler-activation-feedstock/ +* Activation (Windows): https://github.com/conda-forge/clang-win-activation-feedstock/ + +Aside from the main C/C++ compilers, these are the feedstocks for other compilers: +* [CUDA] https://github.com/conda-forge/nvcc-feedstock (CUDA infra currently being overhauled) +* [Rust] `Activation`_ + and `Implementation`_ +* [Go] https://github.com/conda-forge/go-feedstock + +To upgrade the compiler version of our default compilers in the global pinning for +Linux or OSX, ensure that all the above feedstocks have been rebuilt for the respective +new major version, that all interrelated versions are lifted at the same time, +and obviously that the compilers work (e.g. by testing them on some feedstocks by +specifying the new version through the feedstock-local `conda_build_config.yaml`). + +For Windows, we stay on older compilers for longer, because using a newer toolchain +would force everyone wanting to locally develop with conda-forge artefacts to use +a toolchain that's at least as new. A more in-depth explanation can be found +`here`_. CentOS ``sysroot`` for ``linux-*`` Platforms ---------------------------------------------