Skip to content
This repository has been archived by the owner on Sep 18, 2023. It is now read-only.

[NSE-185] Avoid unnecessary copying when simply projecting on fields #187

Merged
merged 2 commits into from
Mar 26, 2021

Conversation

zhztheplayer
Copy link
Collaborator

This is to fix inefficient columnar project executions brought by PR #151 . Other queries that result in field projections may also be accelerated (untested).

@github-actions
Copy link

#185

@github-actions
Copy link


@zhztheplayer zhztheplayer changed the title [NSE-185] Avoid unnecessary copying when simply projecting on fields [DNM][NSE-185] Avoid unnecessary copying when simply projecting on fields Mar 24, 2021
@zhztheplayer
Copy link
Collaborator Author

This would be problematic when a project emits multiple columns from the same column. Say:

SELECT a, a from tab

A fix is needed

@github-actions
Copy link


@zhouyuan zhouyuan changed the title [DNM][NSE-185] Avoid unnecessary copying when simply projecting on fields [NSE-185] Avoid unnecessary copying when simply projecting on fields Mar 25, 2021
@zhztheplayer
Copy link
Collaborator Author

zhztheplayer commented Mar 25, 2021

@zhouyuan I've verified this with TPC-DS Q1~Q20 and TPC-DS Q67. I'm running a complete TPC-DS test now and I think we can merge/backport this anytime we think is needed. And any further troubles I'll log to here and we can then perform rollback.

@zhouyuan zhouyuan merged commit 155afa4 into oap-project:master Mar 26, 2021
zhouyuan pushed a commit to zhouyuan/native-sql-engine that referenced this pull request Mar 30, 2021
…ap-project#187)

* [NSE-185] Avoid unnecessary copying when simply projecting on fields

* Avoid sharing buffers in output
zhouyuan added a commit that referenced this pull request Apr 2, 2021
* [NSE-185] Avoid unnecessary copying when simply projecting on fields (#187)

* [NSE-185] Avoid unnecessary copying when simply projecting on fields

* Avoid sharing buffers in output

* [NSE-211] IndexOutOfBoundsException during running TPC-DS Q2 (#212)

Closes #211

* fix missing decimal check (#219)

Signed-off-by: Yuan Zhou <[email protected]>

* [NSE-227]fix issues from codescan (#225)

* fix code scan

Signed-off-by: Yuan Zhou <[email protected]>

* fix code scan

* fix format

Signed-off-by: Yuan Zhou <[email protected]>

* fix wrong patch

Signed-off-by: Yuan Zhou <[email protected]>

* Klockwork issues fix

* fix format

Signed-off-by: Yuan Zhou <[email protected]>

* more fixes

Signed-off-by: Yuan Zhou <[email protected]>

Co-authored-by: Rui Mo <[email protected]>
Co-authored-by: Hongze Zhang <[email protected]>

* allow to config batchsize in hashagg and wscg (#222)

Signed-off-by: Yuan Zhou <[email protected]>

Co-authored-by: Hongze Zhang <[email protected]>
Co-authored-by: Rui Mo <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants