Ingest tools-automations EKS logs into Splunk

[timmc-edx]

In openedx/edx-platform#32671 we've found that edx-platform CI checks fail because GitHub Actions intermittently lose contact with our runners in EKS. We're trying to determine why, but the logs aren't ingested into Splunk, which makes diagnosis difficult.

A/C:

• We have a way of searching tools-automations EKS logs

[timmc-edx]

From Adam:

Arch can also add Splunk to that cluster here: https://github.com/edx/edx-internal/blob/master/argocd/clusters/tools-automations/installed_applications/kustomization.yaml#L30 by adding
two lines like https://github.com/edx/edx-internal/blob/master/argocd/clusters/tools/installed_applications/kustomization.yaml#L19 + cloning https://github.com/edx/edx-internal/blob/master/argocd/applications/splunk-connect/tools.yaml & https://github.com/edx/edx-internal/tree/master/argocd/applications/splunk-connect-secret-injector/tools but I am slightly curious if we could instead try to enable your desired logging flow with New Relic.

[timmc-edx]

We may not need this if we can get what we need from New Relic.