ABI pinning via SONAME in build string

[carterbox]

Checklist

• Used a personal fork of the feedstock to propose changes
• Bumped the build number (if the version is unchanged)
• Reset the build number to 0 (if the version changed)
• Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
• Ensured the license file is being packaged.

NOTES:

https://www.debian.org/doc/debian-policy/ch-sharedlibs.html
conda-forge/conda-forge.github.io#157

[hmaarrfk]

I'm not sure where I should report this bug but it seems related conda-forge/hdf5-feedstock#177 (comment)

[carterbox]

@hmaarrfk, that lint error has a patch in PR already (but it has not been merged). After the patch the lint will be unaffected by wider use of pinnings with build string matches.

[isuruf]

Why not go with the prior art, which is to have a libgfortran5 output? This is what every other binary distro does.

[carterbox]

Why not go with the prior art, which is to have a libgfortran5 output? This is what every other binary distro does.

I have addressed this in the Alternatives section. It is also discussed in one of the linked issues: conda-forge/conda-forge.github.io#157

An upside to this approach which I have added is that you can install multiple versions simultaneously.

[isuruf]

This is not true. There's a {{ name|lower }} output too, so it is backwards compatible. Please see https://github.com/conda-forge/ctng-compilers-feedstock/blob/main/recipe/meta.yaml

[carterbox]

That's a good point! I'll update this section with a better description and example using your suggestion.

[carterbox]

I have added a more detailed example of this alternative. I think it is the best alternative. I assume that its major utility is that it allows for multiple ABIs of a library to be installed into the same environment. Judging whether this is an important feature for conda-forge is beyond my depth.

I prefer the build string approach because it is simpler and because I haven't run into a case when I wanted to install multiple versions of the same library. However, this is just my personal experience.

[carterbox]

Allowing multiple ABI versions of a package simultaneously causes much more work for refactoring a recipe in order to prevent clobbering. I have updated the proposal to better show how splitting a recipe this way would work.

[carterbox]

Related thought: The purpose of the global conda-forge-pinnings/migrations is to keep all packages in sync with respect to the ABI/API of popular packages. Since publishing ABI-in-the-name packages allows installing multiple ABIs of the same package, does that mean that global conda-forge-pinnings would no longer be needed?

If so, which is easier? Maintaining global pinnings or maintaining individual feedstocks to ensure that ABI-in-the-name packages are implemented (correctly).

[isuruf]

I'm not suggesting we do multiple ABIs at the same time. We have both libgfortran and libgfortran5 as run_exports and if a person really wants multiple ABIs, they can do that, but that's not the default. Because both libgfortran and libgfortran5 are dependencies, you can only have one package.

[carterbox]

https://github.com/conda-forge/libidn11-feedstock/blob/main/recipe/meta.yaml

(Another package that uses SONAME in an new output)

[hmaarrfk]

I think i would like to see conda-forge/abseil-cpp-feedstock#46 resolved before moving more pinnings to conda build strings. It seems that this would exacerbate the issue

[carterbox]

I think i would like to see conda-forge/abseil-cpp-feedstock#46 resolved before moving more pinnings to conda build strings. It seems that this would exacerbate the issue

I didn't anticipate that conda was unprepared to do complex matching in the build string. We could end up with some very long build strings:

v{{ so_name_major }}so_{{ mpi_prefix }}_{{ cuda_prefix }}_h{{ PKG_HASH }}_{{ build }}

which various packages in the same environment could try to match in different ways:

foo 1.*     v{{ so_name_major }}so*
foo >=1.1   v{{ so_name_major }}so*{{ mpi_prefix }}*
foo =1.1.*  v{{ so_name_major }}so*{{ cuda_prefix }}*

And apparently the conda solver is not equipped to resolve these patterns right now.

[h-vetinari]

And apparently the conda solver is not equipped to resolve these patterns right now.

It will be, as soon conda/conda#11612 is merged (and then percolates into the ecosystem).