Git pushes to different branch are behind the base branch.

[jamesloosli]

Hi Y'all,

I'm implementing a workflow where my image update automation flows are updating a different target branch vs the source branch. This seems to be working, but the target branch becomes stale very quickly (we have our flux manifests in a monorepo with ~30 services). Because the branch becomes out of date, automated PR's require manual rebase.

My preferred behavior would be for the image-automation-controller to do a fresh clone of the base branch, then apply updates to the current HEAD of that branch and push --force to the target branch.

If this could at least be a setting available in the CRD, that would be fine. Any other smarter workarounds would be appreciated.

[pjbgf]

Hey @jamesloosli thanks for opening this issue. I noticed we have an old issue that proposed this but was closed a while ago: #205.

But it seems that the way IAC works has changed since then. I added this to the backlog and we will take a look once the Git refactoring piece is done.

[cep21]

I am in the same situation with a large kub cluster as well. I wonder if there should be an info message in the controller logs when it cannot push upstream?

We push to branches so we can run some basic CI/CD on the PR at the github level, before auto merging the PR.

One workaround I am thinking about is force deleting the remote branch if it is stale.

[pjbgf]

This becomes even more relevant when using go-git (WIP) with ShallowClones enabled. I will propose a PR to this effect and share an RC image with it.

[pjbgf]

@jamesloosli @cep21 can you please try the RC image below to confirm that it resolves the problem you are experiencing?

ghcr.io/fluxcd/image-automation-controller:rc-842c6d78

[EmilMunksoe]

@pjbgf I think this is still a bug in the controller. I see this behavior right now using version 0.30.0. My checkout branch is main and push branch is production. I have not overwritten the feature flags as this per. documentation seem to not fix the issue;

If .push.branch is present, the specified branch is pushed to at the origin. The branch will be created locally if it does not already exist, starting from the checkout branch. If it does already exist, it will be overwritten with the cloned version plus the changes made by the controller. Alternatively, force push can be disabled by starting the controller with --feature-gates=GitForcePushBranch=false, in which case the updates will be calculated on top of any commits already on the push branch. Note that without force push in push branches, if the target branch is stale, the controller may not be able to conclude the operation and will consistently fail until the branch is either deleted or refreshed. https://fluxcd.io/flux/components/image/imageupdateautomations/#push

FYI @stefanprodan

[kingdonb]

@EmilMunksoe It's not clear from your reply what issue you're seeing under what circumstances, please open a new issue.

You can refer to this issue if you think it is the same issue, but please provide complete details about the situation. Is your push branch stale and is force enabled? There's not enough information in your post to help you here, and this issue report is resolved/stale so not likely to get attention again, unless we have all that information and can easily confirm your report.