Successful backup triggers another backup

[emu42]

Expected Behavior

I have a backup that is configured to run every 15 minutes and so this is what I expect to happen. The backup currently takes between 6 and 7 minutes, so there should be a pause of 8 to 9 minutes in between backups.

Actual Behavior

Immediately after a backup completes, the backup job is triggered once more (~1-2 second after the first is completed). Oftentimes with the same backup number (which is why @pniederlag previously commented about this on #273 ).

Steps to Reproduce the Problem

Setup kubernetes-operator 0.4.0, configure backup interval to 15 minutes.
This is running on Azure (not sure whether this is relevant to the issue).

Additional Info

Custom log output from backup.sh:

----------------------------------------
2020-05-19 08:57:15 UTC Custom backup 7952 START
2020-05-19 09:03:59 UTC Custom backup DONE
----------------------------------------
2020-05-19 09:04:00 UTC Custom backup 7952 START
7952 already existing, SHOULD NOT HAPPEN
2020-05-19 09:10:28 UTC Custom backup DONE
----------------------------------------
2020-05-19 09:12:15 UTC Custom backup 7953 START
2020-05-19 09:18:43 UTC Custom backup DONE
----------------------------------------
2020-05-19 09:18:44 UTC Custom backup 7953 START
7953 already existing, SHOULD NOT HAPPEN
2020-05-19 09:25:07 UTC Custom backup DONE
----------------------------------------
2020-05-19 09:27:15 UTC Custom backup 7954 START
2020-05-19 09:33:41 UTC Custom backup DONE
----------------------------------------
2020-05-19 09:33:43 UTC Custom backup 7954 START
7954 already existing, SHOULD NOT HAPPEN
2020-05-19 09:40:07 UTC Custom backup DONE
----------------------------------------
2020-05-19 09:42:15 UTC Custom backup 7955 START
2020-05-19 09:48:41 UTC Custom backup DONE
----------------------------------------
2020-05-19 09:48:42 UTC Custom backup 7955 START
7955 already existing, SHOULD NOT HAPPEN
2020-05-19 09:55:09 UTC Custom backup DONE
----------------------------------------
2020-05-19 09:57:15 UTC Custom backup 7956 START
2020-05-19 10:03:40 UTC Custom backup DONE
----------------------------------------
2020-05-19 10:12:15 UTC Custom backup 7957 START
2020-05-19 10:18:45 UTC Custom backup DONE
----------------------------------------

• Kubernetes version:

  Output of kubectl version:

Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.9", GitCommit:"500f5aba80d71253cc01ac6a8622b8377f4a7ef9", GitTreeState:"archive", BuildDate:"2020-01-02T17:13:11Z", GoVersion:"go1.12.13", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.8", GitCommit:"126eb499523e5fffc0138e8e2e031787e5ab1943", GitTreeState:"clean", BuildDate:"2020-04-03T17:45:13Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}

• Jenkins Operator version:

0.4.0

I have looked at some of the kubernetes-operator code and I came across this line in backuprestore.go:

186: return bar.Client.Update(context.TODO(), jenkins)
This is called after the backup successfully completes. Could this trigger the unscheduled backup?

[tomaszsek]

Hi @emu42,

If backup takes 6 to 7 minutes you should rise the resources (CPU and memory) for the backup container.

Cheers

[pniederlag]

@tomaszsek thx for your valuable feedback, very much apprciated. After raising the limit from 100m to 600m the backup is taking only ~80 seconds now. But we still see a backup being retriggerd right after it is finished.

[Sig00rd]

Hello everyone. I'm closing this as I've been unable to reproduce the error on the latest released (v0.6.0) version of the Operator. If the problem does still persist, please file up a new bug report with up to date information. Thank you for your contribution!