Implement TPCH Query 2 in TpchQueryBuilder #9825
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Hi @deepthydavis! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at [email protected]. Thanks! |
✅ Deploy Preview for meta-velox canceled.
|
deepthydavis
force-pushed
the
q2_queryplanner
branch
from
395ccc2
to
3b4b258
Compare
deepthydavis
changed the title
deepthydavis
changed the title
Execution Plan StatisticsOutput of PrintPlanWithStats for 4 drivers: |
pramodsatya
reviewed
May 20, 2024
There was a problem hiding this comment.
Thanks for the changes @deepthydavis, LGTM overall, left some comments.
CMakeLists.txt
Outdated
| @@ -98,13 +98,13 @@ option(VELOX_ENABLE_EXAMPLES | |||
| "Build examples. This will enable VELOX_ENABLE_EXPRESSION automatically." | |||
| OFF) | |||
| option(VELOX_ENABLE_SUBSTRAIT "Build Substrait-to-Velox converter." OFF) | |||
| option(VELOX_ENABLE_BENCHMARKS "Enable Velox top level benchmarks." OFF) | |||
| option(VELOX_ENABLE_BENCHMARKS "Enable Velox top level benchmarks." ON) | |||
| option(VELOX_ENABLE_BENCHMARKS_BASIC "Enable Velox basic benchmarks." OFF) | |||
There was a problem hiding this comment.
Please revert changes to this file and enable them for testing only.
There was a problem hiding this comment.
Reverted the changes in CMakeLists.txt
facebook-github-bot
added
the
CLA Signed
|
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks! |
deepthydavis
force-pushed
the
q2_queryplanner
branch
from
bd29c07
to
6e5cbea
Compare
pramodsatya
approved these changes
May 20, 2024
There was a problem hiding this comment.
Thanks for adding the query plan @deepthydavis.
Hi @aditi-pandit, could you please take a look?
|
Notes : Its a bit odd to me that the tableScan timings for Nation jumped as much between the runs changing numDrivers. But the main node contributing to the time difference is again the TableScan for partsupp likely on account of the dynamic filters. |
|
@deepthydavis : Changes look good. |
aditi-pandit
approved these changes
Jun 4, 2024
|
@deepthydavis : Please can you rebase your code and get a clean build for this PR. We can pass on to the Meta team for merge post that. |
deepthydavis
force-pushed
the
q2_queryplanner
branch
from
6e5cbea
to
3ccf32d
Compare
|
Thanks @deepthydavis @kgpai : Another missing micro-benchmark test for merge. |
aditi-pandit
added
the
ready-to-merge
|
@kevinwilfong has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@kevinwilfong merged this pull request in 20470a0. |
|
Conbench analyzed the 1 benchmark run on commit There were no benchmark performance regressions. 🎉 The full Conbench report has more details. |
deepashreeraghu
pushed a commit
to deepashreeraghu/velox
that referenced
this pull request
Jun 13, 2024
Summary: This PR introduces TPC-H Query 2 into the TpchQueryBuilder and extends the TpchBenchmark and ParquetTpchTest to include this query. Additionally, it provides a detailed performance comparison with DuckDB using the Parquet file format and includes the output of PrintPlanWithStats for detailed analysis. Scaling Factor used is 1. Here is the link to the PowerPoint presentation, which contains a detailed description for each driver and thread : https://ibm.box.com/s/sau464qdfac45aainwpj6pyvkvbtlsat ### Performance Comparison Chip: Apple M1 Pro Total Number of Cores: 10 (8 performance and 2 efficiency) Memory: 32 GB The following table summarizes the performance comparison between Velox and DuckDB (with Parquet file format) across various numbers of threads/drivers: | # Num Threads/ Drivers | Velox(ms) | DuckDB(ms) | |:----------------------:|:---------:|:----------:| | 1 | 27 | 88.4 | | 4 | 23 | 84.1 | | 8 | 25 | 82.8 | | 16 | 30 | 84 | Pull Request resolved: facebookincubator#9825 Reviewed By: bikramSingh91 Differential Revision: D58244304 Pulled By: kevinwilfong fbshipit-source-id: 4d216b10b49847ab692a394783bd5c59a59c9eb2
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces TPC-H Query 2 into the TpchQueryBuilder and extends the TpchBenchmark and ParquetTpchTest to include this query. Additionally, it provides a detailed performance comparison with DuckDB using the Parquet file format and includes the output of PrintPlanWithStats for detailed analysis.
Scaling Factor used is 1.
Here is the link to the PowerPoint presentation, which contains a detailed description for each driver and thread : https://ibm.box.com/s/sau464qdfac45aainwpj6pyvkvbtlsat
Performance Comparison
Chip: Apple M1 Pro
Total Number of Cores: 10 (8 performance and 2 efficiency)
Memory: 32 GB
The following table summarizes the performance comparison between Velox and DuckDB (with Parquet file format) across various numbers of threads/drivers: