Add python (et al.) to used variants even if only referenced via special selectors #5139
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Package-dependent special selectors like
py
and similar are hard-coded to be always available no matter ifpython
etc. are actually recognized as used variant variables.This is problematic since the values of those selectors are not well-defined in case the associated package is not pinned down as variant.
E.g., for a list of variants with different
python
versions, thepy
selector can be any of those versions and is in fact non-deterministically chosen in the current implementation.We have instances in the ecosystem where a multi-output recipe does not have a dependency on
python
but someskip: true # [py ...]
selector is used, as in; e.g., see conda-forge/conda-smithy#1782 and the linked recipes in there.
There are multiple ways to argue about this:
py
withoutpython
at the top level.py
(etc.) selectors should only be defined if the corresponding packages are used.Re. 1.: Yes, the recipe could be amended to have
skip: true # [python and py<39]
or move theskip
line to each output.Regardless, having an implicit non-deterministic behavior is something we should avoid in any case.
Re. 2.: If the selectors would not be defined, the rendering could gracefully fail with an error.
Changing the code to do this could be difficult since the rendering of the current recipe format is rather convoluted.
Plus, there could be recipes that expect those selectors to always be defined, i.e., it might be more disruptive regarding backwards compatibility to remove them.
The alternative, which is implemented here, is to pull in the packages into the "used variant variables" whenever such selectors are used.
This is also a breaking change (due to potentially introducing more variants), but less disruptive as the selectors' removals but possibly (e.g., for the top-level
skip: true # [py<39]
example above) even what the user expected in the first place.N.B.: Personally, I'd consider those selectors as unnecessary tech-debt and hope the ecosystem can move off of their usage with the new recipe format which is currently worked on in the
rattler-build
project.Checklist - did you ...
news
directory (using the template) for the next release's release notes?