Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Go Modules Runs Causes Rate Limit Issues #966

Closed
dmikusa opened this issue Jan 9, 2023 · 0 comments · Fixed by #979
Closed

Update Go Modules Runs Causes Rate Limit Issues #966

dmikusa opened this issue Jan 9, 2023 · 0 comments · Fixed by #979
Labels
hacktoberfest Hacktoberfest eligible note:ideal-for-contribution An issue that a contributor can help us with type:bug A general bug

Comments

@dmikusa
Copy link
Contributor

dmikusa commented Jan 9, 2023

Expected Behavior

Each repo should have a job to update go modules. It should run successfully.

Current Behavior

It will run for most of our repos, but there is a growing number of repos and what happens is that about 10-20% of the repos will fail when the job runs. This is because all of the jobs are configured to run on a cron schedule which kicks off at exactly the same time. This pushes us over Github rate limits.

Possible Solution

Two potential solutions come to mind:

  1. We could use different user tokens. I believe the rate limits are per-user token and all of the jobs run with the same token so as the same user. We could try to break that up and use multiple tokens to spread out the load.
  2. We could stagger the jobs by a small amount. Rather than running them all at 2:00 AM Monday, we could stagger that and have them run sometime randomly (or maybe not randomly, maybe some sort of even distribution) between 2:00AM and 2:30AM.

Steps to Reproduce

https://github.com/paketo-buildpacks/dynatrace/actions/runs/3870244851/attempts/1

Motivations

It is a pain to go and re-run all the failed jobs each week. If that doesn't happen, then updates do not get applied to all of the buildpacks.

@dmikusa dmikusa added type:bug A general bug note:ideal-for-contribution An issue that a contributor can help us with labels Jan 9, 2023
phil9909 added a commit to sap-contributions/pipeline-builder that referenced this issue Jan 20, 2023
phil9909 added a commit to sap-contributions/pipeline-builder that referenced this issue Jan 23, 2023
@anthonydahanne anthonydahanne added the hacktoberfest Hacktoberfest eligible label Oct 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hacktoberfest Hacktoberfest eligible note:ideal-for-contribution An issue that a contributor can help us with type:bug A general bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants