Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

zarf's message when you hit the pull rate limit isn't helpful #2081

Closed
bdw617 opened this issue Oct 17, 2023 · 3 comments · Fixed by #2160
Closed

zarf's message when you hit the pull rate limit isn't helpful #2081

bdw617 opened this issue Oct 17, 2023 · 3 comments · Fixed by #2160
Assignees
Labels
bug 🐞 Something isn't working

Comments

@bdw617
Copy link
Contributor

bdw617 commented Oct 17, 2023

my PC (main, 0.30.1+)

Steps to reproduce

  1. do not be logged into docker.io
  2. run make test-e2e too often

workaround is to simply login to docker.io. but still we shouldn't use it as part of our constantly run testing as much.

Expected result

it should run successfully.

Actual Result

➜ zarf git:(main) ✗ docker pull docker.io/defenseunicorns/zarf-game:multi-tile-dark
Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit

Visual Proof (screenshots, videos, text, etc)

  ⠋  Falling back to docker for docker.io/defenseunicorns/zarf-game:multi-tile-dark. This may take some time.
  ⠋  Falling back to docker for docker.io/defenseunicorns/zarf-game:multi-tile-dark. This may take some time.
  ⠋  Falling back to docker for docker.io/defenseunicorns/zarf-game:multi-tile-dark. This may take some time.
     ERROR:  Failed to create package: unable to pull images after 3 attempts: failed to load metadata for all
             images. This may be due to a network error or an invalid image reference: failed to pull image
             docker.io/defenseunicorns/zarf-game:multi-tile-dark: failed to inspect image
             docker.io/defenseunicorns/zarf-game:multi-tile-dark via docker: Error response from daemon: No such
             image: defenseunicorns/zarf-game:multi-tile-dark

Severity/Priority

Low

@bdw617
Copy link
Contributor Author

bdw617 commented Oct 19, 2023

the CI pipeline got this message today:

 ERROR:  Failed to create package: unable to pull images after 3 attempts: failed to load metadata for all
 images. This may be due to a network error or an invalid image reference: failed to pull image
 public.ecr.aws/docker/library/redis:7.0.11-alpine: failed to pull image
 public.ecr.aws/docker/library/redis:7.0.11-alpine: GET
 https://public.ecr.aws/v2/docker/library/redis/manifests/sha256:ed439ea2fd065ac9622bf40b8bdb156e1e2bb2b73f7877f430775b8d728b3eec:
 TOOMANYREQUESTS: Rate exceeded

@bdw617 bdw617 changed the title test-e2e runs into ratelimiting from docker.io test-e2e runs into ratelimiting from docker.io/CI runs into issues with public.ecr.aws Oct 19, 2023
@bdw617 bdw617 changed the title test-e2e runs into ratelimiting from docker.io/CI runs into issues with public.ecr.aws zarf's message when you hit the pull rate limit isn't helpful Nov 8, 2023
@bdw617
Copy link
Contributor Author

bdw617 commented Nov 8, 2023

In a zarf message:

|      ERROR:  Failed to create package: unable to pull images after 3 attempts: failed to load metadata for all
|              images. This may be due to a network error or an invalid image reference: failed to pull image
|              docker.io/library/httpd:alpine3.18: failed to inspect image docker.io/library/httpd:alpine3.18 via
|              docker: Error response from daemon: No such image: httpd:alpine3.18

if you run docker pull directly:

Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit

it appears to be crane that doesn't expose the message from the server on failure, so the message just says nope, it's not there. we should see if there's a way to improve this, maybe a PR against crane.

@bdw617
Copy link
Contributor Author

bdw617 commented Nov 27, 2023

I've validated where the problem is and it's fixable within zarf. Crane hides the underlying transport info in the error, but Zarf doesn't print out the error from crane unless you have debug on. It's an actual bug, If it can't pull remotely and fails to pull locally, Zarf pull.go loses the original error unless you have debug on.

@bdw617 bdw617 added bug 🐞 Something isn't working and removed possible-bug 🐛 labels Nov 27, 2023
@bdw617 bdw617 self-assigned this Nov 27, 2023
Racer159 added a commit that referenced this issue Dec 15, 2023
…a remote (#2160)

## Description

This fixes a bug where if zarf (via crane) can't pull an image from a
remote repository, and a local tarball doesn't exist, it will output the
proper error from crane, versus hiding the issue as "image does not
exist"


## Related Issue

Fixes #2081 
<!-- or -->
Relates to #

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Other (security config, docs update, etc)

## Checklist before merging

- [x] Test, docs, adr added or updated as needed
- [x] [Contributor Guide
Steps](https://github.com/defenseunicorns/zarf/blob/main/CONTRIBUTING.md#developer-workflow)
followed

---------

Co-authored-by: Wayne Starr <[email protected]>
Co-authored-by: Wayne Starr <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug 🐞 Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant