Support openshift routes

[frzifus]

Closes #902

[frzifus]

actually iam a bit unhappy with the current solution. Any thoughts how to solve this a bit more elegant @pavolloffay ?
Here i try to add and remove a route reconciler depending on the detected platform.

func (r *OpenTelemetryCollectorReconciler) registerOnChangeTask() {
	// NOTE: At the time the reconciler gets created, the platform type is still unknown.
	// TODO: does it actually work? since config is just a copy?
	r.config.OnChange = func() error {
		r.muTasks.Lock()
		defer r.muTasks.Unlock()
		var (
			routesPos = -1
			otelPos   = -1
		)
		for i, t := range r.tasks {
			// search for route reconciler
			switch t.Name {
			case "opentelemetry":
				otelPos = i
			case "routes":
				routesPos = i
			}
		}
		if otelPos == -1 {
			return fmt.Errorf("missing reconciler task: opentelemetry")
		}
		plt := r.config.Platform()
		// if exists and platform is openshift
		if routesPos == -1 && plt == platform.OpenShift {
			end := r.tasks[otelPos:]
			r.tasks = append([]Task{}, r.tasks[:otelPos]...)
			r.tasks = append(r.tasks, Task{reconcile.Routes, "routes", true})
			r.tasks = append(r.tasks, end...)
		}
		// if exists and platform is not openshift
		if routesPos != -1 && plt != platform.OpenShift {
			r.tasks = append(r.tasks[:routesPos], r.tasks[routesPos+1:]...)
		}

		return r.config.OnChange()
	}
}

[frzifus]

ok, it seems the OnChange field is actually never used.

• remove unused onChange function from config #1290

Since we need to register some kind of onChange method to add and remove routes from the reconciliation.. I thought about different ways:

1. the hacky one - here i assume code says more then words Since we are not able to access the onChange methods from outside the config, we would register a function, where we register our reconcile callback afterwards.

	var onChange func() error

	cfg := config.New(
		...
		config.WithOnChange(func() error { return onChange() }),
		...
	)

	...

	onChange := otelReonciler.OnChange

2. Make Config threadsafe and add a RegisterOnChange(f func() error) method to it. Since AutoDetect runs concurrently and while we register something, the internal onChange list could be used we should lock it. The main disadvantage here, is that we would change the passed config to be a ptr in the entire codebase.
3. Another way would be to build some kind of thread-safe OnChangeHandler. Add change handler to register callbacks #1292

Personally, I would prefer solution 3.

[frzifus]

@pavolloffay as discussed in person, an extra structure is introduced to define the route termination.

[frzifus]

@open-telemetry/operator-ta-maintainers thats what i touched there: 5e800c8

[frzifus]

@pavolloffay could you have a look? :)