add explicit securityContexts to the controller and namespace label for privileged

[chrischdi]

What type of PR is this?

/kind feature

What this PR does / why we need it:

This does not really change the configuration, it just makes it explicit and enforce the defaults, except for the seccompPolicy which changes from Unconfined to RuntimeDefault. Syscalls filtered by RuntimeDefault policy are 95% namespaced and require capabilities (which we drop) in the first place, so no practical change there either.

This allows the controller to be compatible to the restricted pod security admission profile.

However, the nmi daemonset requires the privileged profile which is why this also adds the corresponding label to the namespace, to allow cluster-wide enforcement of a policy lower than privileged, while preserving privileged for the capz-system namespace. If the nmi daemonset is replaced by Azure Workload Identity (#2205), and if Azure Workload Identity does not require privileged, the namespace label could get removed again.

This is a recommendation in the CAPI v1.3->1.4 upgrade guide.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

Prior art:

• ✨ add explicit securitycontexts to controllers cluster-api#7831
• add explicit securityContexts to the controller cluster-api-provider-aws#4104

Note: this requires the v1.4 branch of CAPI for usage with tilt, because the Tiltfile then removes the securityContext for allowing it to succeed.

Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

TODOs:

• squashed commits
• includes documentation
• adds unit tests

Release note:

Explicitly set securityContexts in the manifests to comply with the restricted pod security admission profile.

[codecov-commenter]

Codecov Report

Patch and project coverage have no change.

Comparison is base (c125458) 51.64% compared to head (aff2982) 51.64%.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3399   +/-   ##
=======================================
  Coverage   51.64%   51.64%           
=======================================
  Files         182      182           
  Lines       18066    18066           
=======================================
  Hits         9331     9331           
  Misses       8208     8208           
  Partials      527      527           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[CecileRobertMichon]

Thank you @chrischdi

If the nmi daemonset is replaced by Azure Workload Identity (#2205), and if Azure Workload Identity does not require privileged, the namespace label could get removed again.

@sonasingh46 @aramase do you know if this is true?

[Jont828]

This looks good to me and it seems to be passing the tests as well which gives me confidence that it works. I think if we get a second look from others we should be good to proceed.

[aramase]

Thank you @chrischdi

If the nmi daemonset is replaced by Azure Workload Identity (#2205), and if Azure Workload Identity does not require privileged, the namespace label could get removed again.

@sonasingh46 @aramase do you know if this is true?

NMI does need privileged: true in order to add the iptable rules. Azure Workload Identity webhook doesn't require privileged, so this is correct.

[CecileRobertMichon]

/lgtm
/assign @sonasingh46

[CecileRobertMichon]

@sonasingh46 can we make a note to remove this in #2924 ?

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: 9105c70aee9a030d21c99e091efcabaffd75a87f

[jackfrancis]

cc @sonasingh46 if you're able to add another review to this in the 1.9 milestone cycle that would be great!

[sonasingh46]

Sure. I will take a look at this tomorrow.

[mboersma]

/lgtm

[mboersma]

/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mboersma

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [mboersma]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment