Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fallback to solved record filename to find the downloaded tarball in get_upstream_pins #5037

Merged
merged 2 commits into from
Oct 18, 2023

Conversation

jaimergp
Copy link
Contributor

@jaimergp jaimergp commented Oct 12, 2023

Description

Closes #4991

After quite some effort debugging this issue, this is what we've found:

  • The channel configuration must either redefine default_channels or channel_alias, or both.
  • The package cache must be clean (e.g. after conda clean -pt)

In this situation, the get_upstream_pins calculation is forced to download the package on the spot to extract it and retrieve its run_exports metadata.

The problem is caused by a small divergence in the indices (parsed repodatas) as loaded by libmamba and conda, respectively. With customized default_channels and channel_alias values, the channel attribute of each PackageRecord in the conda-build index is set differently than in libmamba.

For example, with this configuration:

channels:
  - defaults
default_channels:
  - conda-forge
solver: libmamba

, the offending block tries to match these two records:

  • conda-forge::python-3.11.6-h43d1f9e_0_cpython (provided by libmamba, the solver)
  • defaults::python-3.11.6-h43d1f9e_0_cpython (calculated by conda-build's index, which is built separately but not used in practice)

They are the same package with different channel. Since conda-build loads the repodata on its own, it interprets that the conda-forge channel should be called defaults given the config in default_channels. However that's a multichannel and libmamba doesn't handle those well and it prefers to use the actual channel behind the multichannel abstraction (e.g. pkgs/main for the default defaults). Hence the divergence.

I was going to try fix this in conda-libmamba-solver so the indices match, but:

  • That's going to be a bigger fix touching an already fragile interface
  • The fix in conda-build is insultingly simple. I can't imagine a scenario where the filenames may not match, and I would say the must by design. That means we can reuse the one in the solver-provided PackageRecord. I am only adding it as a fallback if not found in the conda-build index as an abundance of caution.
  • This code path will be obsolete soon when we add run_exports.json support (via CEP-11).

Checklist - did you ...

  • Add a file to the news directory (using the template) for the next release's release notes?
  • Add / update necessary tests?
  • Add / update outdated documentation?

@conda-bot conda-bot added the cla-signed [bot] added once the contributor has signed the CLA label Oct 12, 2023
kenodegard
kenodegard previously approved these changes Oct 16, 2023
@kenodegard kenodegard enabled auto-merge (squash) October 18, 2023 14:24
@kenodegard kenodegard merged commit f7cf445 into conda:main Oct 18, 2023
21 checks passed
@kenodegard kenodegard mentioned this pull request Dec 1, 2023
67 tasks
@github-actions github-actions bot added the locked [bot] locked due to inactivity label Oct 18, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 18, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
cla-signed [bot] added once the contributor has signed the CLA locked [bot] locked due to inactivity
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

KeyError when using libmamba solver with conda-build
3 participants