Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create checksum_mismatch.yml #922

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Create checksum_mismatch.yml #922

wants to merge 1 commit into from

Conversation

dward4
Copy link

@dward4 dward4 commented Jan 18, 2024 •
edited
Loading

Both of these packages have a checksum mistmach and do not download and install.

Guidelines for marking packages as broken:

  • We prefer to patch the repo data (see here)
    instead of marking packages as broken. This alternative workflow makes environments more reproducible.
  • Packages with requirements/metadata that are too strict but otherwise work are
    not technically broken and should not be marked as such.
  • Packages with missing metadata can be marked as broken on a temporary basis
    but should be patched in the repo data and be marked unbroken later.
  • In some cases where the number of users of a package is small or it is used by
    the maintainers only, we can allow packages to be marked broken more liberally.
  • We (conda-forge/core) try to make a decision on these requests within 24 hours.

What will happen when a package is marked broken?

  • Our bots will add the broken label to the package. The main label will remain on the package and this is normal.
  • Our bots will rebuild our repodata patches to remove this package from the repodata.
  • In a few hours after the anaconda.org CDN picks up the new patches, you will no longer be able to install the package from the main channel.

Checklist:

  • I want to mark a package as broken (or not broken):

    • Added a description of the problem with the package in the PR description.
    • Pinged the team for the package for their input.

  • I want to archive a feedstock:

    • Pinged the team for that feedstock for their input.
    • Make sure you have opened an issue on the feedstock explaining why it was archived.
    • Linked that issue in this PR description.
    • Added links to any other relevant issues/PRs in the PR description.

  • I want to request (or revoke) access to an opt-in CI resource:

    • Pinged the relevant feedstock team(s)
    • Added a small description explaining why access is needed

@dward4
Create checksum_mismatch.yml
ab58810 
Both of these packages have a checksum mistmach and do not download and install.
@dward4 dward4 requested a review from a team as a code owner January 18, 2024 19:17
@beckermr
Copy link
Member

We need to report this to anaconda inc maybe as well?

@mbargull
Copy link
Member

mbargull commented Jan 18, 2024 •
edited
Loading

Both of these packages have a checksum mistmach and do not download and install.

Works for me. All checksums I can think of match and installation works:

# curl -sLO 'https://conda.anaconda.org/conda-forge/win-64/xorg-libxdmcp-1.1.3-h301d43c_0.tar.bz2'
# curl -sLO 'https://conda.anaconda.org/conda-forge/win-64/pcre2-10.40-h17e33f8_0.tar.bz2'
# sha256sum *.tar.bz2
5833c63548e4fae91da6d77739eab7dc9bf6542e43f105826b23c01bfdd9cb57  pcre2-10.40-h17e33f8_0.tar.bz2
34e2798cd98c78683f2a7f272d9c30e5a0c30206f9eb043ae78bb52b873fdf11  xorg-libxdmcp-1.1.3-h301d43c_0.tar.bz2
# md5sum *.tar.bz2
2519de0d9620dc2bc7e19caf6867136d  pcre2-10.40-h17e33f8_0.tar.bz2
3eb2af45e531888ead3d80f85ff825df  xorg-libxdmcp-1.1.3-h301d43c_0.tar.bz2
# curl -sL 'https://conda.anaconda.org/conda-forge/win-64/repodata.json' > repodata.json
# <repodata.json jq -r '.["packages"]["pcre2-10.40-h17e33f8_0.tar.bz2"]|[.sha256, .md5]'
[
  "5833c63548e4fae91da6d77739eab7dc9bf6542e43f105826b23c01bfdd9cb57",
  "2519de0d9620dc2bc7e19caf6867136d"
]
# <repodata.json jq -r '.["packages"]["xorg-libxdmcp-1.1.3-h301d43c_0.tar.bz2"]|[.sha256, .md5]'
[
  "34e2798cd98c78683f2a7f272d9c30e5a0c30206f9eb043ae78bb52b873fdf11",
  "3eb2af45e531888ead3d80f85ff825df"
]
# curl -sL 'https://conda.anaconda.org/conda-forge/win-64/repodata.json.bz2' | bzip2 -d > repodata.json
# <repodata.json jq -r '.["packages"]["pcre2-10.40-h17e33f8_0.tar.bz2"]|[.sha256, .md5]'
[
  "5833c63548e4fae91da6d77739eab7dc9bf6542e43f105826b23c01bfdd9cb57",
  "2519de0d9620dc2bc7e19caf6867136d"
]
# <repodata.json jq -r '.["packages"]["xorg-libxdmcp-1.1.3-h301d43c_0.tar.bz2"]|[.sha256, .md5]'
[
  "34e2798cd98c78683f2a7f272d9c30e5a0c30206f9eb043ae78bb52b873fdf11",
  "3eb2af45e531888ead3d80f85ff825df"
]
# curl -sL 'https://conda.anaconda.org/conda-forge/win-64/repodata.json.zst' | zstd -d > repodata.json
# <repodata.json jq -r '.["packages"]["pcre2-10.40-h17e33f8_0.tar.bz2"]|[.sha256, .md5]'
[
  "5833c63548e4fae91da6d77739eab7dc9bf6542e43f105826b23c01bfdd9cb57",
  "2519de0d9620dc2bc7e19caf6867136d"
]
# <repodata.json jq -r '.["packages"]["xorg-libxdmcp-1.1.3-h301d43c_0.tar.bz2"]|[.sha256, .md5]'
[
  "34e2798cd98c78683f2a7f272d9c30e5a0c30206f9eb043ae78bb52b873fdf11",
  "3eb2af45e531888ead3d80f85ff825df"
]
# mkdir pcre2-10.40-h17e33f8_0
# bsdtar -xC pcre2-10.40-h17e33f8_0 -f pcre2-10.40-h17e33f8_0.tar.bz2
# ( cd pcre2-10.40-h17e33f8_0 && <info/paths.json jq -r '.paths[]|[._path, .sha256]|join("\n")' | xargs -n2 -- sh -c 'sha256sum "${1}" | grep -qF "${2}" || echo check failed for "${1}"' -- )
# mkdir xorg-libxdmcp-1.1.3-h301d43c_0
# bsdtar -xC xorg-libxdmcp-1.1.3-h301d43c_0 -f xorg-libxdmcp-1.1.3-h301d43c_0.tar.bz2 
# ( cd xorg-libxdmcp-1.1.3-h301d43c_0 && <info/paths.json jq -r '.paths[]|[._path, .sha256]|join("\n")' | xargs -n2 -- sh -c 'sha256sum "${1}" | grep -qF "${2}" || echo check failed for "${1}"' -- )
# CONDA_SUBDIR=win-64 conda create -qynp pcre2=10.40=h17e33f8_0
Channels:
 - conda-forge
Platform: win-64
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: /opt/conda/envs/p

  added / updated specs:
    - pcre2==10.40=h17e33f8_0


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    bzip2-1.0.8                |       hcfcfb64_5         122 KB  conda-forge
    libzlib-1.2.13             |       hcfcfb64_5          54 KB  conda-forge
    pcre2-10.40                |       h17e33f8_0         1.9 MB  conda-forge
    ucrt-10.0.22621.0          |       h57928b3_0         1.2 MB  conda-forge
    vc-14.3                    |      hcf57466_18          17 KB  conda-forge
    vc14_runtime-14.38.33130   |      h82b7239_18         732 KB  conda-forge
    vs2015_runtime-14.38.33130 |      hcb4865c_18          17 KB  conda-forge
    ------------------------------------------------------------
                                           Total:         4.1 MB

The following NEW packages will be INSTALLED:

  bzip2              conda-forge/win-64::bzip2-1.0.8-hcfcfb64_5 
  libzlib            conda-forge/win-64::libzlib-1.2.13-hcfcfb64_5 
  pcre2              conda-forge/win-64::pcre2-10.40-h17e33f8_0 
  ucrt               conda-forge/win-64::ucrt-10.0.22621.0-h57928b3_0 
  vc                 conda-forge/win-64::vc-14.3-hcf57466_18 
  vc14_runtime       conda-forge/win-64::vc14_runtime-14.38.33130-h82b7239_18 
  vs2015_runtime     conda-forge/win-64::vs2015_runtime-14.38.33130-hcb4865c_18 


Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
#
# To activate this environment, use
#
#     $ conda activate p
#
# To deactivate an active environment, use
#
#     $ conda deactivate

# CONDA_SUBDIR=win-64 conda create -qynx xorg-libxdmcp=1.1.3=h301d43c_0
Channels:
 - conda-forge
Platform: win-64
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: /opt/conda/envs/x

  added / updated specs:
    - xorg-libxdmcp==1.1.3=h301d43c_0


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    m2w64-gcc-libgfortran-5.3.0|                6         342 KB  conda-forge
    m2w64-gcc-libs-5.3.0       |                7         520 KB  conda-forge
    m2w64-gcc-libs-core-5.3.0  |                7         214 KB  conda-forge
    m2w64-gmp-6.1.0            |                2         726 KB  conda-forge
    m2w64-libwinpthread-git-5.0.0.4634.697f757|                2          31 KB  conda-forge
    msys2-conda-epoch-20160418 |                1           3 KB  conda-forge
    xorg-libxdmcp-1.1.3        |       h301d43c_0          65 KB  conda-forge
    ------------------------------------------------------------
                                           Total:         1.9 MB

The following NEW packages will be INSTALLED:

  m2w64-gcc-libgfor~ conda-forge/win-64::m2w64-gcc-libgfortran-5.3.0-6 
  m2w64-gcc-libs     conda-forge/win-64::m2w64-gcc-libs-5.3.0-7 
  m2w64-gcc-libs-co~ conda-forge/win-64::m2w64-gcc-libs-core-5.3.0-7 
  m2w64-gmp          conda-forge/win-64::m2w64-gmp-6.1.0-2 
  m2w64-libwinpthre~ conda-forge/win-64::m2w64-libwinpthread-git-5.0.0.4634.697f757-2 
  msys2-conda-epoch  conda-forge/win-64::msys2-conda-epoch-20160418-1 
  xorg-libxdmcp      conda-forge/win-64::xorg-libxdmcp-1.1.3-h301d43c_0 


Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
#
# To activate this environment, use
#
#     $ conda activate x
#
# To deactivate an active environment, use
#
#     $ conda deactivate

@mbargull mbargull marked this pull request as draft January 18, 2024 19:50
@dward4
Copy link
Author

dward4 commented Jan 23, 2024

I'm seeing the expected checksum now as well via certiutil:

certutil -hashfile .\win-64_pcre2-10.40-h17e33f8_0.tar.bz2 SHA256
SHA256 hash of .\win-64_pcre2-10.40-h17e33f8_0.tar.bz2:
5833c63548e4fae91da6d77739eab7dc9bf6542e43f105826b23c01bfdd9cb57

I tried a fresh install of miniconda, made a new environment and i'm still getting errors:

ChecksumMismatchError: Conda detected a mismatch between the expected content and downloaded content
for url 'https://conda.anaconda.org/conda-forge/win-64/pcre2-10.40-h17e33f8_0.tar.bz2'.
download saved to: C:\Users<SSO>\AppData\Local\miniconda3\pkgs\pcre2-10.40-h17e33f8_0.tar.bz2
expected sha256: 5833c63548e4fae91da6d77739eab7dc9bf6542e43f105826b23c01bfdd9cb57
actual sha256: 2816cfa047dc46376c52dd5a33fa526b23703c9aea1e6ca6e7f4c77d00dc0b40

ChecksumMismatchError: Conda detected a mismatch between the expected content and downloaded content
for url 'https://conda.anaconda.org/conda-forge/win-64/pcre2-10.40-h17e33f8_0.tar.bz2'.
download saved to: C:\Users<SSO>\AppData\Local\miniconda3\pkgs\pcre2-10.40-h17e33f8_0.tar.bz2
expected sha256: 5833c63548e4fae91da6d77739eab7dc9bf6542e43f105826b23c01bfdd9cb57
actual sha256: 2816cfa047dc46376c52dd5a33fa526b23703c9aea1e6ca6e7f4c77d00dc0b40

Do you have guidance on where to go next? I couldn't find how conda executes checksums. Thank you for all your help!

@jaimergp
Copy link
Member

Do you have guidance on where to go next? I couldn't find how conda executes checksums. Thank you for all your help!

It's at https://github.com/conda/conda/blob/545e3c82222cf4b774eadd64c7c2a4a6b25066a1/conda/gateways/disk/read.py#L71, essentially delegating to the Python stdlib.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@dward4 @beckermr @mbargull @jaimergp