Level driven controller

[maiqueb]

What this PR does / why we need it:
Re-write the controller as level driven, instead of edge-triggered.

This makes user interaction more resilient, since the controller will try to materialize the desired state (network-selection-elements) into the current state (network-status).

There is one behavior that needs to be highlighted: the existing interfaces need to always feature the interface name in the desired state (network selection elements); take the following example:

• initially, there is a secondary interface in the pod for which the user did not name explicitly.
  • at t=1, the user attempts to hotplug interfaces; these will fail
  • at t=2, the user corrects the desired state, manually specifying the name of the interface for the existing attachment
  • the controller will reconcile the pod state, and will now be able to hotplug the new interfaces, specified at t=2

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #48
Fixes #66

Special notes for your reviewer (optional):
Depends-on: #46

[kmabda]

The approach looks good to me.

I do see though we're missing handling the case where an update is made to the networks annotation while the controller is not present:

• t=1 while the controller is not started or crashed or restarting after a rollup upgrade etc., a user changed a running pod annotation to hotplug an interface
• t=2 controller start

To reconcile, on startup we need to do call pnc.podsLister.List() to add all Running pods on the controller's node that are not in the hostNetwork to the pnc.workqueue

[maiqueb]

The approach looks good to me.

I do see though we're missing handling the case where an update is made to the networks annotation while the controller is not present:

• t=1 while the controller is not started or crashed or restarting after a rollup upgrade etc., a user changed a running pod annotation to hotplug an interface
• t=2 controller start

To reconcile, on startup we need to do call pnc.podsLister.List() to add all Running pods on the controller's node that are not in the hostNetwork to the pnc.workqueue

Good point; would you mind opening an issue and pushing a PR to address this @kmabda ? I do not know exactly where that instruction should be put.