Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[idea] full-aggregate pushdown under Gather #745

Open
kaigai opened this issue Mar 25, 2024 · 1 comment
Open

[idea] full-aggregate pushdown under Gather #745

kaigai opened this issue Mar 25, 2024 · 1 comment
Labels
feature developer considers it is the feature to be supported on the future version
Milestone

Comments

@kaigai
Copy link
Contributor

kaigai commented Mar 25, 2024

非常にパーティション数が多く、GROUP BYによってパーティションキーの一つ以上が指定されている場合。

Final Aggregate
 + Gather
    + Append
          + Partial Aggregate on Partition 1
          + Partial Aggregate on Partition 2
          + Partial Aggregate on Partition 3
                       :
          + Partial Aggregate on Partition N

こういった場合、AppendやGatherに加え、最後のFinal Aggregationの負荷も非常に大きい。

しかし、Partition 1を扱うプロセス(ワーカー)が一個だけである事を保証できれば、Final Aggregationは不要になる。
クエリの特性とパーティションキーの配置を考えて、こういった実行計画を作れるようにしたい。

Gather
 + Append
    + Aggregate on Partition 1
    + Aggregate on Partition 2
                :
    + Aggregate on Partition N

やる事

  • GpuPreAggのCPU-Fallback --> インフラストラクチャとして必要
  • この最適化を発動できないケースの整理

メリット

  • PostgreSQLが苦手な重複排除系のクエリで効果を発揮しやすいハズ
@kaigai kaigai added the feature developer considers it is the feature to be supported on the future version label Mar 25, 2024
kaigai added a commit that referenced this issue May 16, 2024
A groundwork for #752, #748, #745, #738.
This fix allows to back fallback tuple with l_state individually.
It enables to restart execution of only fallback'ed tuples, thus,
removed technical problems to implement CPU fallback of GpuPreAgg.
@kaigai kaigai modified the milestones: v5.3, v6.0 Dec 25, 2024
@kaigai
Copy link
Contributor Author

kaigai commented Dec 25, 2024

Phase.1 は Partition-wise GpuJoin なし(v5.3)、
Phase.2 では Partition-wise GpuJoin あり(v6.0)で。

kaigai added a commit that referenced this issue Jan 2, 2025
related to issue #745, needs to drop & create extension;
kaigai added a commit that referenced this issue Jan 2, 2025
kaigai added a commit that referenced this issue Jan 2, 2025
@kaigai kaigai modified the milestones: v6.0, v5.3 Jan 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature developer considers it is the feature to be supported on the future version
Projects
None yet
Development

No branches or pull requests

1 participant