feat: Implement config pod status

[abhipatnala]

What this PR does / why we need it:

Which issue(s) this PR fixes (optional, using fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged):
Fixes #2918

Special notes for your reviewer:

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 48.97959% with 125 lines in your changes missing coverage. Please review.

Project coverage is 48.22%. Comparing base (3350319) to head (20cb2ca).
Report is 157 commits behind head on master.

Files with missing lines	Patch %	Lines
apis/status/v1beta1/zz_generated.deepcopy.go	0.00%	62 Missing ⚠️
...controller/configstatus/configstatus_controller.go	64.47%	17 Missing and 10 partials ⚠️
pkg/controller/config/config_controller.go	66.15%	15 Missing and 7 partials ⚠️
apis/status/v1beta1/configpodstatus_types.go	80.00%	2 Missing and 2 partials ⚠️
pkg/controller/expansion/expansion_controller.go	50.00%	2 Missing and 2 partials ⚠️
...er/constraintstatus/constraintstatus_controller.go	0.00%	1 Missing and 1 partial ⚠️
...platestatus/constrainttemplatestatus_controller.go	0.00%	1 Missing and 1 partial ⚠️
...ller/expansionstatus/expansionstatus_controller.go	0.00%	1 Missing and 1 partial ⚠️

❗ There is a different number of reports uploaded between BASE (3350319) and HEAD (20cb2ca). Click for more details.

HEAD has 1 upload less than BASE

Flag	BASE (3350319)	HEAD (20cb2ca)
unittests	2	1

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3544      +/-   ##
==========================================
- Coverage   54.49%   48.22%   -6.28%     
==========================================
  Files         134      221      +87     
  Lines       12329    15371    +3042     
==========================================
+ Hits         6719     7413     +694     
- Misses       5116     7119    +2003     
- Partials      494      839     +345     

Flag	Coverage Δ
unittests	48.22% <48.97%> (-6.28%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[maxsmythe]

It looks like selfOnly is not being used. This code was copied from expansion templates, which also did not use selfOnly. This is a bug. Here is an example of proper use of selfOnly in the constraint controller:

gatekeeper/pkg/controller/constraint/constraint_controller.go

Lines 194 to 198 in 07127f2

	err = c.Watch(
	source.Kind(mgr.GetCache(), &constraintstatusv1beta1.ConstraintPodStatus{}, handler.TypedEnqueueRequestsFromMapFunc(constraintstatus.PodStatusToConstraintMapper(true, util.EventPackerMapFunc()))))
	if err != nil {
	return err
	}

Here is the basic reasoning:

• status controllers want to watch all podStatus objects and the primary object because they want to make sure they respond to any changes (i.e. writing status changes, overwriting inappropriate deletes of the status field, etc.)

• primary controllers want to watch podStatus for the corresponding pod -- if someone deletes a podStatus resource the main object should be re-reconciled to avoid missing state.

As it stands, if a user were to delete the podStatus object, there is a risk that there would be no reconcile.

We should add the appropriate watches to the config and expansion controllers.

[abhipatnala]

Updated.

[maxsmythe]

Could we have a PR that fixes this in the other controller?

[ritazh]

If selfOnly is always required and true, why is this a variable? when does it need to be set to false?

[maxsmythe]

It doesn't look like we're gating status controller on whether the status operation is enabled -- this is bad because it means this controller will not run as a singleton, which invites write conflicts particularly as the # of pods scales.

Example of the status gate:

gatekeeper/pkg/controller/constrainttemplate/constrainttemplate_controller.go

Lines 155 to 178 in 07127f2

	if operations.IsAssigned(operations.Status) {
	// statusEvents will be used to receive events from dynamic watches registered
	// via the registrar below.
	statusEvents := make(chan event.GenericEvent, 1024)
	csAdder := constraintstatus.Adder{
	CFClient: cfClient,
	WatchManager: wm,
	ControllerSwitch: cs,
	Events: statusEvents,
	IfWatching: statusW.IfWatching,
	}
	if err := csAdder.Add(mgr); err != nil {
	return nil, err
	}
	ctsAdder := constrainttemplatestatus.Adder{
	CfClient: cfClient,
	WatchManager: wm,
	ControllerSwitch: cs,
	}
	if err := ctsAdder.Add(mgr); err != nil {
	return nil, err
	}
	}

Though a more appropriate code shape for this PR is probably the mutator status gate:

gatekeeper/pkg/controller/mutatorstatus/mutatorstatus_controller.go

Lines 60 to 62 in 07127f2

	if !operations.IsAssigned(operations.MutationStatus) {
	return nil
	}

(however we should depend on Status, not MutatorStatus)

We should also fix this for the expansion template status controller (worth verifying it has a similar oversight first).

Long-term a more uniform design pattern for adding status controllers may help avoid similar oversights in the future, but that's beyond the scope of this PR.

[abhipatnala]

Updated the logic to use status gate

[maxsmythe]

Can we create PRs to add it to the other status controllers that do not have a similar gate? This would include expansion template status at the minimum.

[maxsmythe]

almost there! Minor nits and some follow-up items.

[maxsmythe]

Can we create PRs to add it to the other status controllers that do not have a similar gate? This would include expansion template status at the minimum.

[maxsmythe]

Could we have a PR that fixes this in the other controller?

[maxsmythe]

Code LGTM, pending resolution of either fixing readiness tracker tests in this PR or separating code to separate PR.

[maxsmythe]

LGTM after final minor fix. Good job!

[maxsmythe]

LGTM

[JaydipGabani]

Thanks for the PR! changes looks good to me.

One q:
Issue mentiones adding gatekeeper_config metrics and status tag for the same as well. Do we want to include those changes in this PR or keep the original issue open and follow up with another PR?

[maxsmythe]

Let's make it a separate PR. This one is already sizeable and that's conceptually a different thing. Maybe file a separate issue for metrics and status tag?

[JaydipGabani]

LGTM

[ritazh]

This overrides existing rbac rules per https://github.com/open-policy-agent/gatekeeper/blob/master/pkg/controller/config/config_controller.go#L130-L131

should the old ones be removed?

[maxsmythe]

I think what it does is adds overlapping entries in the role rather than override?

IMO it's safest to just duplicate to avoid accidentally deleting necessary permissions due to code refactors, but not a huge deal either way for me.

[ritazh]

I'm ok with keeping it. Just wanted to make sure this is what we want as it trumps the old rules.

groups=config.gatekeeper.sh,resources=*,verbs=get;list;watch;create;update;patch;delete

[ritazh]

LGTM