Extract funnel steps directly in the events subquery #13922
Labels
feature/funnels
Feature Tag: Funnels
performance
Has to do with performance. For PRs, runs the clickhouse query performance suite
team/product-analytics
Currently we pass all of an event's properties from the innermost funnels subquery to be parsed into the steps further up. The problem with this is that we then pipe all that data over the network from one shard to another, whereas we could push a lot of the computations down to the individual shards and only send the results over.
I've done some initial benchmarking on this and the results are promising:
https://metabase.posthog.net/dashboard/24-query-benchmarking?query_tag=funnels-84510088626212
https://metabase.posthog.net/dashboard/24-query-benchmarking?query_tag=funnels-29424020244
Just look at the difference in cross-shard data transfer!
The change is reasonably simple from the perspective of the final query, but it's a bit of an involved refactor given how complicated the funnels query building logic is.
Writing this issue down to document things but I'm already working on it.
Another thing is that I think we can also remove a few levels of nesting when it comes to breakdowns, but that's something I need to also dig into.
The text was updated successfully, but these errors were encountered: