Add prometheus scrape annotations to agent

[objectiser]

Following on from #27, we should add the prometheus scrape/port annotations to the agent.

Issue with the agent as sidecar is to ensure we don't overwrite similar annotations provided by the service.

[annanay25]

What would be the solution when the service and the agent would like to expose different ports for Prometheus scraping? @objectiser

[objectiser]

Not sure - I created the issue as a reminder to check, but unfortunately haven't yet had time to experiment. There is a prometheus issue discussing the topic with one suggested workaround that requires changing the prometheus scrape config which isn't ideal.

[annanay25]

I agree @objectiser

Can we change the agent config to publish to the same port as the service?

[objectiser]

Don't think we can have two processes using the same port within the same pod.

[jpkrohling]

This should be a good first issue, I think. All work would probably be located in pkg/inject/sidecar.go, most likely inside the decorate() function. In case the deployment already has Prometheus annotations, the new logic would just be skipped. Otherwise, new annotations would be added:

		Annotations: map[string]string{
			"prometheus.io/scrape":    "true",
			"prometheus.io/port":      "5778",
		},

The tests for this should be easy to write as well: a positive case, asserting that a deployment without annotations will get the annotations once the sidecar is injected, and a negative case, where deployments with the annotations won't be touched at all (scrape=false, port=1212, for example)