Add ephemeral-storage resource requests/limits to Flux Deployments

[MartinEmrich]

Describe the bug

The Flux Deployments currently lack ephemeral-storage requests and limits. This makes the pods more prone to eviction if a node gets DiskPressure, and increases the likelyhood of DiskPressure in the first place. https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#local-ephemeral-storage

If all other scheduled pods specify their storage needs, this will also throw off the scheduling.

Steps to reproduce

1. install flux
2. check the deployment (e.g. kubectl -n flux-system get deployment notification-controller -oyaml)
3. the resources: have cpu and memory, but not ephemeral-storage

Expected behavior

Set some reasonable values (logs to stdout/stderr also count towards ephemeral storage.

If the Flux controllers don't need emptyDir storage, I would try a request/limit of 64/128MiB, that should fit the default log rotation scheme (https://kubernetes.io/docs/concepts/cluster-administration/logging/#log-rotation). If they do need an emptyDir, the values might be higher depending on the requirements.

Screenshots and recordings

No response

OS / Distro

Debian 12

Flux version

2.4.0

Flux check

► checking prerequisites
✔ Kubernetes 1.31.4 >=1.28.0-0
► checking version in cluster
✔ distribution: flux-v2.4.0
✔ bootstrapped: false
► checking controllers
✔ helm-controller: deployment ready
► ghcr.io/fluxcd/helm-controller:v1.1.0
✔ image-automation-controller: deployment ready
► ghcr.io/fluxcd/image-automation-controller:v0.39.0
✔ image-reflector-controller: deployment ready
► ghcr.io/fluxcd/image-reflector-controller:v0.33.0
✔ kustomize-controller: deployment ready
► ghcr.io/fluxcd/kustomize-controller:v1.4.0
✔ notification-controller: deployment ready
► ghcr.io/fluxcd/notification-controller:v1.4.0
✔ source-controller: deployment ready
► ghcr.io/fluxcd/source-controller:v1.4.1
► checking crds
✔ alerts.notification.toolkit.fluxcd.io/v1beta3
✔ buckets.source.toolkit.fluxcd.io/v1
✔ gitrepositories.source.toolkit.fluxcd.io/v1
✔ helmcharts.source.toolkit.fluxcd.io/v1
✔ helmreleases.helm.toolkit.fluxcd.io/v2
✔ helmrepositories.source.toolkit.fluxcd.io/v1
✔ imagepolicies.image.toolkit.fluxcd.io/v1beta2
✔ imagerepositories.image.toolkit.fluxcd.io/v1beta2
✔ imageupdateautomations.image.toolkit.fluxcd.io/v1beta2
✔ kustomizations.kustomize.toolkit.fluxcd.io/v1
✔ ocirepositories.source.toolkit.fluxcd.io/v1beta2
✔ providers.notification.toolkit.fluxcd.io/v1beta3
✔ receivers.notification.toolkit.fluxcd.io/v1
✔ all checks passed

Git provider

No response

Container Registry provider

No response

Additional context

(A valid option would be to just provision more storage, but this might not always be a wise (or possible) choice in embedded or cost-sensitive environments)

Code of Conduct

• I agree to follow this project's Code of Conduct