Split off metrics

[chadwhitacre]

What do you think about splitting up the ci-tooling repo so that metrics lives by itself? My concern is that the slack/github/freight changes can end up breaking metrics

[chadwhitacre]

We have a thread over on the OSS side to move at least some things from actions into an app: getsentry/.github#5. This could be the beginning of that. I.e., first split out the OSS metrics pipeline into an app owned by Open Source instead of p10y, then we can start adding write/push features such as getsentry/team-ospo#2 getsentry/.github#56 getsentry/.github#55.

Thoughts @BYK?

[chadwhitacre]

(cc: @billyvg @armenzg in case you're not already seeing this.)

[billyvg]

IMO metrics should live in an app by itself, independent of any automations, otherwise we'll run into the same concerns as the OT. Metrics in this case, in addition to OSS metrics, would also include sentry + getsentry + Freight metrics as well.

For the OSS/GH automations, you may or may not want to piggy back off the existing infrastructure in this repo.

[chadwhitacre]

@billyvg Are you envisioning two separate repos, or separate apps in the same repo?

[BYK]

@chadwhitacre my understanding is two separate repos sharing some core infra/logic. That said I still strongly oppose an app we run and manage (instead of github actions). The only exception is if we setup Google Cloud Run and deploy on merge to master automatically so it is essentially the same thing, but on Google Infra :D

[chadwhitacre]

If the opposition to an app instead of actions is ease of deployment then yeah let's figure out easy deployment. I think it is worth shifting to an app for the greatly improved reaction time. I want to apply e.g. a Team: * label and see the bot respond "immediately" rather than waiting a couple minutes. Provides a better experience, knowing the bot is working and noticing sooner when it fails.

[billyvg]

It should be 2 separate GH Apps as we can properly configure the webhook endpoints + subscribe to the relevant events/permissions. In regards to repos, I have less of an opinion on that, but separate may make things easier as we would need better tooling to support a monorepo.

And FWIW, this repo auto deploys to GCR on master pushes, so that infra is already in place.

[chadwhitacre]

We're definitely not a monorepo org so it doesn't seem like we should go that route here.

[chadwhitacre]

Okay! So let's fork this repo—sentry-dev-metrics?—and rip metrics out of this one and rip everything else out of the new one. Sound right?

[chadwhitacre]

I guess the real question is who wants to take this on? :) Do you have bandwidth @billyvg?

[billyvg]

I can work on it but it won't be until later on in the quarter (or next quarter).

[chadwhitacre]

We decided against this. We're going to mitigate the original concern by making sure that metrics happen early in the request/response cycle.