Sprint 1.46.0 1/2 - March 6 to March 17

[lharries]

Global Sprint Planning

3 things that might take us down

1. Running out of integers @fuziontech - still have lots of time
2. Bad migrations - Tim already pushed the PR to be stricter on tests
3. General scaling issues - will want to do proper capacity planning later on
4. Rolling out lightweight capture tomorrow morning, first on EU then on US. Removes postgres dependency. Keep an eye out for anything strange

Support hero this sprint

Week 1: Eric
Week 2: Harry

Team sprint planning

For your team sprint planning copy this template into a comment below for each team.

Team ___

## Retro

<!-- Grab the high and low priority items from last time and add whether that item was completed or not -->

- 

## Hang over items from previous sprint

<!-- For each item, decide to re-prioritise (and add below) or deprioritise -->

- Item 1. prioritised/deprioritise

## OKR

1. OKR, status (red/yellow/green) and action points if yellow/red


### High priority

-

### Low priority / side quests

-

[mariusandra]

Team Analytic, Product

Retro & hang over

• Data Exploration @thmsobrmlr @pauldambra

  • Converting the last missing components. Data fetching via dataNodeLogic
  • Dashboards with queries on them, insight active view.
  • Retro: Solution exists for all hard things. TODO: the work 😅. Lots of sidequests and frustration, but we're pretty much there for queries on insights/dashboards.

• PostHog 3000 @Twixes

  • Nail the layout paradigm.
  • Retro: Spent more time on visual regression tests. ChartJS is annoying. 3000 coming along -> work is progressing on the layout.

• HogQL @mariusandra

  • Direct SQL querying in beta
  • Retro: In alpha for our team. Will get there late this week or early next.

• Performance @macobo @yakkomajuri

  • Map and clarify high level potential wins
  • Wrap up all the refresh work.
  • Retro: pretty good. Sampling is nice. PoE will improve accuracy somewhat, at the expense of a huge migration.

• Extra bits:

  • React library from @lharries
  • Dashboards templates also from @lharries

OKR & Strategy

OKRs

• Objective 1: Ship PostHog 3000 UX. 10 happy ICP beta customers.
• Objective 2: Make PostHog performance frustration free for our 10 largest customers
• Objective: Systematically prevent regressions across PostHog Part 2

Strategy

• Improve the core UX to be more focused on product engineers:
  • A slick experience
  • More powerful querying than our competitors can offer (for example, SQL access) that answers the long tail of questions
  • PostHog 3000 UX = a design uplift including dark mode to encourage more word of mouth

Sprint planning 🏮🏮🏮

People

• Time off:
  • Offsite in Munich next week for session recording. Thomas might join some days.
• Support primary/secondary:
  • TBD

Plan next sprint

• HogQL @mariusandra

  • Missing filters (person belongs to cohort) and odds and ends.
  • Full ClickHouse SQL compatibility, including arrays, unions, lambdas, etc.
  • Why? Last bits left to be able to build posthog on top of hogql.

• Data Exploration & PostHog 3000 @thmsobrmlr @Twixes

  • Funnels step table and correlations into data exploration.
  • Insights list based on PostHog 3000 designs.
  • Why? Slicker nicer interface that sparks joy.

• Performance, queries, cancellation, caching @yakkomajuri @pauldambra

  • Insight sampling into beta.
  • Cancellation and caching for insights and queries.
  • Why? Features over benefits.

Sidequests / parking lot

There's a volvo in the parking lot.

[alexkim205]

Team Recordings

Retro

• Decided not to work on Notes on Recordings because work might overlap with other solutions we're thinking about for PostHog 3000.

Hang over items from previous sprint

• @benjackwhite MVP of alternative ingestion for Session Recordings
  why? - If we can get this running in parallel with our current system, at least for one team (us) we can validate a lot of our initial assumptions (cost, effort etc.)
  • feat(recordings): add separate deployments/hpa for ingestion charts-clickhouse#701

OKR

O1: Eliminate the biggest reasons product engineers would choose a competitor over PostHog Session Recording
🟡 KR1: Release iOS Mobile Recordings to 5 happy beta testers
🟢 KR2: Release the ‘Network Tab’ to 10 happy customers
O2 Recordings work and are available
🟢 KR1: Rollout RRWeb2 and improve our ability to fix / improve
🔴 KR2: Move to new S3-backed storage for cheaper and longer storage

High priority

@annikaschmid @benjackwhite @alexkim205 will be in Munich together March 6-10.

• @benjackwhite Continue with MVP of alternative ingestion for session recordings
• @benjackwhite Rollout alpha iOS recordings
• @alexkim205 Work on Android recordings library <- pending meetup agenda

Low priority / side quests

• Have fun in Munich

[EDsCODE]

Team Feature Flag/Experiment

Retro

Eliminate the biggest reasons product engineers would choose a competitor over PostHog Feature Flags

• No developments here. Mostly addressed in first half of sprint

Make the experience of creating a feature flag in PostHog slicker

• Provide ready to go analysis after flag deployment—We have a motley of floating tools that users can use to analyze flags but they’re slow/unclear at the moment (insights on flags, recordings on flags) @EDsCODE
  • Usage tab WIP. Can be shipped end of week
• Address feature flag instrumentation clarity—There are a handful of ways that flags can be used and not a lot of guidance in app on how to handle different situations @liyiy
  • Onboarding Wizard WIP. Can be shipped end of week
• Making a decision on condensing or changing api @neilkakkar
  • Decision should be made by end of week

Planning

Make the experience of creating a feature flag in PostHog slicker @liyiy

• Rest of UI suggestions left in the figma

Eliminate the biggest reasons product engineers would choose a competitor over PostHog Feature Flags @neilkakkar

• Cohort Local Evaluation in Node and Python
• Resiliency updates

Analysis of feature flags using recordings @EDsCODE

• Finish optimizing recordings list query
• Figuring out how to filter more accurately for recording associated with flag
  • Neil: “send something like feature_flag_called everytime the flag call happens”

[tiina303]

🚰 Team Pipeline 🚰

Retro

• @fuziontech - spicy .. 🔥s feeling optimistic about the future, maybe because of the ❄️ hanging out with eric, tim, james was 🎉
• @hazzadous - "great"
• @tiina303 - the way we communicate with customers about exports and replicator worries me
• @xvello - looking forward to Frank onboarding so infra has more engineering time ; lightweight capture going well 🙂 CI is becoming annoying with flaky tests, GH issues 🙁
• @tomasfarias - excited to get PoE out & then 🔥killed my excitement 🙁

Previous goals

• 🔴Historical exports: make them prod-ready: confirm recent fixes helped, improve monitoring, add ability to stop exports (wishlist: resume them on failure)
  • why: a pain point for customers (slowness, duplicates), while the feature is table-stakes
  • update: Change of plans, RFC for v3 coming soon
• ✅ Lightweight capture (remove PG dependency): enable token resolution at plugin-server for all messages, phase out PG lookup at capture, still keep some token validation at capture (best effort PG lookup, or just validate the shape?)
  • why: reduce impact of the second prod-us Postgres switch next month
  • update: 🤞done by EOW 🎉 Would have helped even more with outages had we rolled this out sooner.
• 🔴Person on Events?
  • why: Gotta go fast!
  • update: Model changes roll-out triggered an incident 🔥, write path to team 2 likely won't ship by EOW, but we have a plan to move forward.

Next sprint

Support secondary (Luigi):
week 1: Xavier
week 2: Harry (also support hero)

Goals:

• PoE “Person Overrides” write path shipped for teams who already use PoE @tiina303
  • why: Query Performance while maintaining Data Quality
• Session Recording ingestion lag automatically handled and stop the unbounded lag growth @hazzadous
  • why: We're currently manually scaling up, even 64 pods (partitions) have a hard time keeping up
• RFC strategy for the future & manual exports for high priority customers now & @fuziontech
  • why: Exports are important for customers and we want this to be a good experience

OKRs

Objective: Performance

• 🟡 Key Results: We have wrapped up the person-on-event project and have deprecated the old non-person-on-events queries
  • Why? Performance speed up
  • Progress This is the top priority for the team again so progress will happen, but we likely won't complete the full original KR.
• ✅🟠 Key Results: We have reduced the cost per event for capture by an order of magnitude
  • Why? Infra savings and improves performance
  • Progress Lightweight capture will be rolled out by EOW. No further work planned.

Objective: Reliability

• 🟢 Key Results: We have converted all current US dashboards into IaC dashboards configured in Terraform and made all necessary migrations from StatsD to Prom to support this.
  • Why? Gets US and EU equivalent in terms of monitoring
  • Progress Not much progress this sprint as we worked on Lightweight capture.
• 🔴 Key Results: All of our alerts have runbooks
  • Why? Improve incident recovery times and share knowledge with all engineers, so that most incidents can be resolved without escalating to the team
  • Progress Not started, unlikely we'll do anything here.
• 🟠 Key Results: Backfills do not slow us down or take down the system. We have tests for this.
  • Why? Improves service quality and protects against bad actors
  • Progress Plugin-server has hot partition overflow now for events, which has already helped avoid pages and lag. Last step we're planning to do is enabling it from capture directly.
• 🔴 Key Results: Erroring apps fail gracefully, do not take down anything else, and we have tests to prove this. And re-enable after temporary unavailability
  • Why? Improves service quality and tackles customer annoyance of apps turning off when there's an error
  • Progress Not started, unlikely we'll do anything here.

[raquelmsmith]

Team CA$H 💸

Retro

• Finished up rate limiting but have not started dropping events yet
• Shipped email verification, haven't launched to customers yet
• Fixed many issues in billing service: missing reports, tools for CS, etc
• Don't get sick this sprint!
• Make sure we're paying attention to our project board in Github
• Billing emails - hope to have a PR by friday

High priority

• Decide how we are going to go about moving from plans -> products pricing
• Start on the work for the above

Low priority

• Billing emails @kappa90
  • Will finish up this sprint
  • Create the logic to send the emails @kappa90
  • Create the email templates @raquelmsmith
• Make autocapture toggleable in the UI @raquelmsmith
• Fix customer.io plugin and allow reporting stats back to PostHog @kappa90

OKR

1. Objective 1: Feel confident in our definitions and metrics for all our areas of responsibility

   • Status: 🍏
   • Brand new Growth dashboard, with links to sub-dashboards: https://app.posthog.com/dashboard/54139

2. Objective 2: Improve conversion to paid

   • Status: 🍎
   • Metric we set at the beginning is not relevant (we don't know where the number came from)
   • Currently at ~1% conversion rate from signup to paid, and ~3% from just activation (Discoveries) to paid
   • Action point: run the experiments we have planned, all work done so far is going to enable experimenting much faster

3. Objective 3: We can iterate quickly on pricing

   • Status: 🍋
   • We're changing how we go about pricing from plans -> products so we have to redo things

[ellie]

Infra Team

@danielxnj is infra hero this sprint

OKRs

No major changes since last time

Carryovers

• Finish migrating to VictoriaMetrics from Prometheus (dashboards + alerting switched over, vmagent etc) @ellie
• Setup read replicas on Aurora + use them in our app @ellie

Plan

• Ensure Aurora postgres replica is robust and reliable @danielxnj
• Replace proxy headers + setup CDN @danielxnj
• Clear up documentation as people are struggling to access things @ellie
• Onboard @frankh! 🥳 @ellie