-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Schedule does not seems to be respected #3059
Comments
The schedule is ~best effort, and it's roughly when we start update jobs, it might take a while for PRs to actually be opened. There are also a few other reasons why a PR could be opened, for example a security advisory was published, and when a job has previously failed we also re-run it when a change to the manifest file has been pushed. So what you're seeing could be explained by one of the above, could you share a bit more details about what you're seeing, and maybe share the repo name so I can check some logs etc? |
Repo is a private (company one) The logs in the dependabot change didn't start too far from 6PM (7PM) yesterday 2021-02-02T06:59:06Z Additionally All the PR got rebased at about 3PM (3hours before schedule) forcing our build pipeline to start building a big amount of open PR (those massive rebase were not in the dependabot changelog). |
👍
Without knowing the repo it's hard to check up on this, an hour later than scheduled is not great, but it can happen due to a lot of back pressure. Feel free to open a ticket via support so it's easier to share details if you'd like us to dig deeper!
By default, PRs get rebased when dependabot detects conflicts, but this can be disabled: https://docs.github.com/en/github/administering-a-repository/configuration-options-for-dependency-updates#rebase-strategy |
Since you recommend to search for similar issues before posting one, with my company we have the same issue for one private repository. We have something like ~30 repositories for Javascript projects, all of them sharing the same configuration (see below) which basically schedules 2 upgrades maximum every week. But only for one repository Dependabot keeps creating upgrades as soon as there is one available. The initial limit has already been processed long time ago. No matter what we do (recreating the file for instance) Dependabot creates an upgrade, at any time. The shared configurationversion: 2
updates:
# Required options
- package-ecosystem: npm
directory: "/"
schedule:
interval: weekly
day: saturday
time: "00:00"
timezone: Europe/Paris
# Behaviour of pull requests
open-pull-requests-limit: 2
pull-request-branch-name:
separator: "-"
commit-message:
prefix: build
include: scope
# Metadata of pull requests
reviewers:
- "xxx/yyy"
# Control which dependencies are updated
ignore:
- dependency-name: "ember-cli" |
I have a PR that I can share here: openhab/openhab-android#2590 version: 2
updates:
- package-ecosystem: github-actions
directory: "/"
schedule:
interval: daily
time: "04:00"
timezone: Europe/Berlin
open-pull-requests-limit: 99
- package-ecosystem: gradle
directory: "/"
schedule:
interval: daily
time: "04:00"
timezone: Europe/Berlin
open-pull-requests-limit: 99
Since there was another PR one minute earlier and none of them have something about security in their release notes, I don't think that's a security relase: openhab/openhab-android#2588 |
I'm having the same issue. I setup two repos with the following dependabot.yml configuration: version: 2
updates:
- package-ecosystem: "npm"
directory: "/"
schedule:
interval: "weekly"
day: "monday"
ignore:
- dependency-name: "*"
update-types: ["version-update:semver-patch"] For some reason, the PRs get opened right after I close any of the 5 open PRs. The schedule isn't respected at all and PRs are just opened whenever one is closed. Any ideas why that may be? |
Maybe because of the default pr limit? Can you try to increase it as in my config above? |
Ok I applied the limit, i'll report back with status |
Looks like that just kept on popping PRs up, still not respecting the schedule (weekly on Monday's) today is Sunday |
I have an example too. This should run daily at 6am, but for some reason - hasn't run for the last 2 days - I know there is a package to update too... It is a public repo: https://github.com/samsmithnz/AzurePipelinesToGitHubActionsConverterWeb/blob/main/.github/dependabot.yml
|
Having the exact same issue as @guatedude2. IssueAs soon as we merge 1 PR, Dependabot creates a new one, effectively keeping us busy reviewing Dependabot updates all week. What I'd expectI'd expect Dependabot to respect the schedule and only offer updates on Sunday, 19:00 with a maximum of 10 Dependabot PR's open at the same time As a work-around, we have disabled patch and minor updates for now, but this is of course FAR from ideal. We have a private repository. Our
|
This ⬆️ |
The behavior as described by others still persists. It's almost as if Dependabot has its own internal queue of dependency updates (which is filled on the scheduled interval), but as soon as we merge or close a Dependabot PR, it will create the next one from its queue. The desired behavior is that it only creates new (non-security update) PRs at the scheduled time, and not creates more PRs out-of-bound after closing previous PRs. The one exception to this would be when Dependabot is newly added to a project, or a new package ecosystem is added to the Dependabot config file. The reason we use Dependabot is to allocate a roughly fixed amount of time each week to perform updates, to have an upper bound on the amount of time we spend on our maintenance and technical debt each week. Right now the out-of-bound Dependabot PRs clutter our PR list, which adds an annoyance factor to an otherwise awesome product. |
What I think is happening here, is that dependabot is triggered when your manifest files (package.json for example) change, and kicks off a new job. Normally when this happens during the week for a manual change to those files, dependabot will not do anything because the open PR limit is reached, however when this happens when a PR is merged, once that job is finished, Dependabot will notice a "free" slot in that limit, and open a PR. I get this is annoying and unexpected, I'm just not immediately sure of a good way to fix it. We might be able to disable the behavior of re-running dependabot on a manifest file change when we notice that the person who changed it was Dependabot, but that might be slightly tricky, as a merged PR will show the author as the person that merged 🤔 |
Thank @jurre, that looks in line with what @asciimike describes in #3975 (comment) We define the scheduled interval specifically to have one weekly moment where new Dependabot PRs are created. It's on Sunday night for us, so CI can run as long as it needs and we can review dependency PRs on Monday morning. We cherry pick the Dependabot PRs into one new branch/PR and perform some additional work (applying new linting rules, grabbing all We thought that we could control the behavior via What you describe re: creating new Dependabot PRs for existing package ecosystems in response to manifest file changes is explicitly an anti-goal for us. Looking at manifest changes to close or update existing PRs is expected behavior. Security updates already follow their own out-of-bounds schedule, so that's no reason for the behavior either. What's the reasoning for looking at manifest file changes and opening new PRs, in context of there being a schedule and PR limits? If your intention behind the PR limits to have an unlimited "queue" of PRs Dependabot wants to create, but to simply limit the number of PRs which can exist at the same time? In this case maybe we need an additional In the PR I mentioned I saw that the team raising the issue switched to Renovate over this. I can tell you that members of our team have mentioned them as an alternative to look at over this exact issue as well. Personally I like to see what we can do to help you solve the issue and stay on Dependabot, so I'm happy to help to try to think of solutions. |
@Narnach could you share the repo this is happening on? I'd like to run some checks |
@jurre it's for reisbalans/reisbalans, a private repo |
Thanks! After digging deeper I think an update is only supposed to happen if the last update failed. The reasons there being that the failure is possibly resolved after a manifest file change. So this shouldn't happen every time you merge a PR 🤔 I do see a few update jobs that have errors in the logs for that repository, could that explain what's happened? Or does it consistently happen every time even when there are no errors? If you can share the pull request number for one of the PRs that was created after one of these merges that'd help a lot (so, |
Dependabot consistently fails the final step of the linting check, because it tries to report annotated feedback to Github. I think Dependabot does not have sufficient rights to annotate our code, hence that job fails ( It works when our team members initiate the check. We figured it was due to Github tightening security measures over the last few months. Because we create a new PR for the actual merge, it's been a low priority to look into it. The most recent dependency PR we merged was: It created these PRs in response:
|
This is a CI check you're referring to? That should be fine and ignored, we only care about the dependabot itself failing (you should be able to see this on https://github.com/reisbalans/reisbalans/network/updates).
I don't have access to this repo, but in the Dependabot DB I cannot find a reference to that PR, so am I right in assuming this was not created by us? Was the dependabot config file changed in that PR by any chance? |
Sorry, I thought you meant CI check logs 😅 The network updates check failures appear to be for hitting the limit on open PRs: package.json has a limit of 5 that's reached, and we've got a secondary Gemfile in a subdirectory for an embedded project which has been set to limit 0 to ignore it (seemed the best way to get Dependabot to ignore a manifest file 🤷 ) PR 6041 was indeed created by me, and is the aggregate dependency update PR for this week. I did update the Dependabot config there to make Dependabot ignore a gem which is pinned on a specific version (and will be phased out rather than updated soon), but the |
That should be fine and not count as an "error" state, that's just Dependabot working as intended and not opening more PRs than what's configured.
Ah, any change to the config file triggers updates immediately. Our assumption is that users will want these changes applied immediately in most cases. I think that explains what may have happened here, or can you think of other cases where this definitely was not the culprit? The only thing I can recommend right now is to separate changes to the config file out and apply them before merging/closing any of Dependabot's PRs. This behavior is documented here fwiw, and I think it makes sense for most users. I suppose we could add an option to disable this, but to be honest I don't think our small team currently has the bandwidth to tackle it any time soon. |
Checking back I see evidence to support this theory (internal PRs):
Having an option to disable unscheduled non-security PR creation would probably be the most easily understandable option for users. I did a quick search in the code, but scheduling does not appear to be part of the open source code, so I don't think I can help out with a PR for this. I hope your team manages to pick it up at some point. I'll see if our increased understanding of Dependabot's behavior makes it less annoying for us. Thanks for the informative conversation! |
Thanks @Narnach
cc @asciimike, for future reference
Yeah that's right, this is all part of the services we use to run Dependabot at GitHub which are all private.
I hope it does! 💛
Thanks for helping me figure out what happened here! |
@jurre I've just processed this week's dependency updates, but after merging my PR we got 5 new Dependabot PRs. My PR did not contain changes to My dependencies PR: After merging the PR, we got these additional PRs from Dependabot:
We did get a PR for an update to a Github Action, so that resulted in some of the workflow files in |
I don't think so, the code in question checks against this constant: CONFIG_FILE_PATHS = %w(.github/dependabot.yml .github/dependabot.yaml).freeze But thanks for these links, I'll try to find some time to figure out what happened here! |
[Update: at first I thought this was only for the One side-effect of this I've seen a number of times over the last 2-3 days, is that when dependabot creates a new PR to replace a closed one, sometimes those PRs appear to use stale state. Here's an example (unfortunately in a private repo, but at least illustrative): Thu Jul 29 19:25:59 2021 +0000, I merge a dependabot PR to "Bump ngx-infinite-scroll from 9.1.0 to 10.0.1". While this is probably somewhat harmless in many cases, there are a couple of things that I find really annoying about this:
|
I've got an ongoing conversation with the PR team about the "Dependabot PRs are in a stale state", though unfortunately it's not moving quickly. It's not specific to Dependabot, but Dependabot users happen to hit it a lot more frequently than the average user. Your comment about rebasing to resolve the "bump x and y" is the right course of action, though I agree it's a pain :/ |
I'm seeing the same as other people and interestingly the same phenomenon as @rafalyesware - hadn't thought anything of this when I noticed it before. I just closed a PR for This issue makes trying to maintain a WIP limit impossible and turns managing dependabot prs into a game of whack-a-mole. |
I'm also having the same issue, has there been any update on a fix for this? |
All my team's Gradle projects see this same behavior because of consistent Dependabot update errors triggered by #3759. I would love to see the latter fixed to remove such errors, though I think the current immediate-retry-after-error-if-manifest-or-config-file-changed update logic is fine. Otherwise (or in addition), another way to make Dependabot update errors more visible would be welcome. |
I am seeing this same behavior with gradle projects. Schedule is not respected, and often new PRs get created soon after the old ones are merged. |
Same issue here for |
Same problem, every time I close a PR a new one get's created. Config looks something like this:
|
Incredible how maintainers keep giving us feedback on this incorrect behaviour. |
Adding to this thread.
version: 2
updates:
- package-ecosystem: bundler
insecure-external-code-execution: allow
directory: "/backend"
schedule:
interval: monthly
time: "07:00"
timezone: America/Vancouver
open-pull-requests-limit: 99
registries:
- rubygems-server-gems-graphql-pro |
Folks, you can also use Renovate instead: I feel like Dependabot is becoming an abandonware at this point. |
@MrChocolatine thanks for sharing that, I've never seen that before so I'll evaluate it for our team. Out of curiosity, why do you say Dependabot is becoming abandonware? Obviously this issue is concerning, but wondering if you have some more info. |
I may be completely wrong, I am just a simple user.
I am not claiming it is becoming an abandonware, but I feel like. Again, it's a personal feeling, perhaps I am expecting too much from this service. |
👋 Hey @MrChocolatine! I'm the PM for Dependabot. I'm sorry to hear that you feel like there's lack of news or feedback. This particular issue hasn't seen much traction because it's not one of our top priorities. But if you look at #1190 or #1736, we're working on some of the top requested features from the community, and actively seeking feedback. GitHub's a pretty big company so sometimes news about our little bot can get lost in the noise. But if you keep an eye on our tag on the GitHub changelog, you can see all the stuff we're releasing. I'd be happy to chat further and hear your feedback - feel free to select the time most convenient for you from my calendar: https://calendar.app.google/ZqyPQoHMuT6tsKVX9 |
I am glad to be wrong then, thank you for this message. |
Would it be possible to at least get a configuration option to ignore changes to manifest files and only trigger either on schedule or when the dependabot config is changed ? As it stands the schedule feature seems like it's pretty much useless |
@raphael-theriault-swi ; @deivid-rodriguez is working on this and we hope to have this resolved this week :). Thanks for being patient and raising this up. |
Hei! We deployed an internal fix for this. Please let us know if you receive any more PRs out of schedule from now on. (Note that when you edit your |
@deivid-rodriguez I;'m still getting weird schedule problems with dependabot: version: 2
updates:
- package-ecosystem: "npm"
directory: "/"
schedule:
interval: "monthly"
While configured to monthly, it runs several times a month. Any input on that? |
The only situation when Dependabot runs out of schedule now is when the |
Hi, I'm trying to use the schedule parameter with time: "18:00" and timezone: "Pacific/Auckland" but it seems like dependabot doesn't respect the 18:00 for some reasons ? Am I doing something wrong ?
The text was updated successfully, but these errors were encountered: