🗣️ Add Team Drilldown in Looker

[hubertdeng123]

Teams that are really good at engaging with users will be more successful because feedback is priceless. It is the raw material for our product vision. If we don’t engage, feedback dries up, and we build a product our customers don’t find useful.

We’ve been reaching 95% issue SLO response time for Support - great! We’ve been at like 80-85% for POs - good! How do we get POs to great? We don’t have good enough data to generate and validate hypotheses. So we want to improve our visibility. We want teams to have a clear idea of the product areas they own, and we want to filter down our GitHub response metrics to track the performance of individual teams. This will help surface data in order to find learning opportunities on what we’re doing well and what we can do better.

More Code as a Source of Truth

Last quarter we started using the product-owners.yml in security-as-code as a source of truth for defining product owners for product areas. However, it’s not well defined how these product areas map to internal teams.

Each product area will no longer be mapped to product owners, but a team. One team can own multiple product areas. This will clear up any ambiguity on the relationship between product areas and teams. If a product area grows to span multiple teams, it will be broken down into multiple product areas.

Teams will now be defined in security-as-code and will map to a list of product owners and the slack channel to send notifications to. Sentry repos will also be defined in security-as-code to map to product areas. This enables us to develop visibility into the teams and product areas that are associated with each repo.

Example snippet of future product-owners.yml

by_team:
  team-issues:
    product_owners: # populates GitHub teams
      - eran.arkin
      - rachel.wang
      - zpropersi
      - brian.thi
      - scefali
    slack_channel: foo 
  team-dnd:
    product_owners:
      - adam.mckerlie
      - brent.charbonneau
    slack_channel: bar 

by_area:
  "Sign In":
    - "team-enterprise"
  "Issues":
    - "team-issues"

by_repo:
  sentry-javascript: "team-web-sdks"
  relay: "team-ingest"
  self-hosted: "team-ospo"
  sentry-native: "team-processing"

Dashboard Changes

We’ll need to overhaul the 401 dashboard. It will require us to enable filtering the issue response by issues owned by a team instead of all issues. This will allow teams to have a more fine tuned dashboard to track how well they’re doing on issue response, and issues that are slipping through the cracks.

Weekly Looker scores to publish a team’s response performance to slack will allow teams to keep track of how well they’re doing, and if there’s work to be done to improve.

Tasks

Preview Give feedback

1. Programmatically require sentry teams and repos to be well defined in product-owners.yml in security-as-code #173
   4 of 4
   +0
2. Automate PR’s in security-as-code to sync with eng-pipes workflows #174
   2 of 2
   +0
3. Milestone 1 Complete
4. Start tracking GH customer engagement by teams #175
   4 of 4
   +0
5. Milestone 2 Complete
6. Enable slack notifications for sdk repos #177
   2 of 5
   +0
7. Slack team scores #176
   2 of 2
   +0
8. Milestone 3 Complete

Future work

• giving incentives for good GH response times

[hubertdeng123]

Weekly Update Aug 7

• https://github.com/getsentry/security-as-code/pull/307
• https://github.com/getsentry/security-as-code/pull/311
• Sync product owners with security-as-code eng-pipes#580

[hubertdeng123]

Investigated moving slack team scores to eng-pipes, think I should be able to query data from bigquery to get slack weekly scores into eng-pipes. Testing this locally doesn't seem to be an easy task though.

[chadwhitacre]

Notifications are working! 💃

[chadwhitacre]

Weekly reports are also working! 💃