feat: Add support for Hubble control plane in Retina agent

[anubhabMajumdar]

Description

This PR adds support for Hubble control plane in Retina agent. This is being done in the most backward compatible way possible.

I am adding a new subcommand called hubble-control-plane which will start Hubble instead of existing control plane.

$  ./retina -h                                                                                                                                 ✔  took 5s 
Start Retina Agent

Usage:
  retina-agent [flags]
  retina-agent [command]

Available Commands:
  completion           Generate the autocompletion script for the specified shell
  help                 Help about any command
  hubble-control-plane Start Hubble control plane

Flags:
      --config string                      config file (default "/retina/config/config.yaml")
      --health-probe-bind-address string   The address the probe endpoint binds to. (default ":18081")
  -h, --help                               help for retina-agent
      --leader-elect                       Enable leader election for controller manager. Enabling this will ensure there is only one active controller manager.
      --metrics-bind-address string        The address the metric endpoint binds to. (default ":18080")

Use "retina-agent [command] --help" for more information about a command.
$
$
$  ./retina                                                                                                                                                ✔
Starting Retina Agent
starting Retina daemon with legacy control plane
...
$
$
$ ./retina hubble-control-plane -h
Start Hubble control plane

Usage:
  retina-agent hubble-control-plane [flags]
  retina-agent hubble-control-plane [command]

Available Commands:
  hive        Inspect the hive

Flags:
      --cluster-name string                        name of the cluster (default "default")
...
$
$
$ ./retina hubble-control-plane --config-dir .
ts=2024-06-13T18:20:47.888Z level=info caller=hubble/daemon_main.go:276 msg="Traces telemetry initialized with zapai" version= appInsightsID=
time="2024-06-13T18:20:47Z" level=info msg=Invoked duration="483.869µs" function="pprof.init.func1 (pkg/pprof/cell.go:49)" subsys=hive
time="2024-06-13T18:20:47Z" level=info msg=Invoked duration="77.011µs" function="gops.registerGopsHooks (pkg/gops/cell.go:38)" subsys=hive
time="2024-06-13T18:20:47Z" level=info msg=Invoked duration=10.503879ms function="github.com/microsoft/retina/cmd/hubble.init.func3 (cmd/hubble/daemon.go:68)" subsys=hive
time="2024-06-13T18:20:47Z" level=info msg="&{{ 0}  [] 0s false true false false false false}" subsys=agent-config
time="2024-06-13T18:20:47Z" level=info msg="configuring telemetry" app-insights-id= retina-version= subsys=telemetry
time="2024-06-13T18:20:47Z" level=info msg="telemetry disabled" subsys=telemetry
ts=2024-06-13T18:20:47.902Z level=info caller=metrics/metrics.go:169 msg="Metrics initialized"
...

Changes made

• Adopting Hive for dependency injection
• Moving to Cobra for CLI
• controller/main.go now is just the starting point of the command
• retina/cmd now houses rootCmd (starts retina as is) and hubble (starts Hubble control plane)
• In terms of Dockerfiles, packaging Hubble cli in agent image
• Adding new YAML files to install Retina with Hubble
• Moved the current YAML files under deploy/legacy
• Fix the links in doc
• Update cilium version to pull in commits from upstream needed for starting Hubble
• Update init to add a step that creates Cilium dirs (This will happen for current control plane as well, but it consumes no resources, just creates an empty directory)
• All new packages under pkg contains business logic required to run Hubble (node reconciler, Hubble control plane, IPCache, etc.)
• Minor changes to test/e2e to support change to deployment directory (deploy -> deploy/legacy)

Related Issue

#418

Checklist

• I have read the contributing documentation.
• I signed and signed-off the commits (git commit -S -s ...). See this documentation on signing commits.
• I have correctly attributed the author(s) of the code.
• I have tested the changes locally.
• I have followed the project's style guidelines.
• I have updated the documentation, if necessary.
• I have added tests, if applicable.

Screenshots (if applicable) or Testing Completed

Retina with Hubble

Retina

Please refer to the CONTRIBUTING.md file for more information on how to contribute to this project.

[huntergregory]

Looks awesome. I mainly had nitpicks about dashboards and deleting unused bits

[matmerr]

🛰️

[huntergregory]

It looks like the UT files are missing in deploy/hubble/grafana/ (can copy from deploy/legacy/grafana/)

The merge-base changed after approval.