feat: cost of for is ~0 #967
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We fairly often perform FoR and then do not perform bitpacking because we have exceeded the depth. FoR is very cheap and only worthwhile in combination with bitpacking anyway, so setting its cost to zero should improve compression ratio with a minor decrease in compresssion speed.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Vortex bytes_at
| Benchmark suite | Current: 601e840 | Previous: 389e6a4 | Ratio |
|---|---|---|---|
bytes_at/array_data |
598.7257088247189 ns (1.6810395754905016) |
652.5428677998441 ns (1.6311862543452094) |
0.92 |
bytes_at/array_view |
877.4696826518971 ns (0.4287715889546462) |
952.1401865492534 ns (0.7060914092799635) |
0.92 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Random Access
| Benchmark suite | Current: 601e840 | Previous: 389e6a4 | Ratio |
|---|---|---|---|
random-access/vortex-tokio-local-disk |
1212307.855957945 ns (5530.50631449034) |
1095867.8831195845 ns (16136.36256233824) |
1.11 |
random-access/vortex-local-fs |
1355142.1107412938 ns (6742.463942734757) |
1214918.8855048325 ns (4947.895539387944) |
1.12 |
random-access/parquet-tokio-local-disk |
197464090.73333332 ns (2063772.931250006) |
194698968.10000002 ns (3657368.312499985) |
1.01 |
This comment was automatically generated by workflow using github-action-benchmark.
robert3005
approved these changes
Oct 2, 2024
lwwmanning
approved these changes
Oct 2, 2024
There was a problem hiding this comment.
DataFusion
| Benchmark suite | Current: 601e840 | Previous: 389e6a4 | Ratio |
|---|---|---|---|
arrow/planning |
813550.3143955758 ns (3137.1342436449486) |
804581.4664068398 ns (838.1301516011008) |
1.01 |
arrow/exec |
1751753.7696704906 ns (1065.6151481547859) |
1749718.9759410908 ns (2217.692730856361) |
1.00 |
vortex-pushdown-compressed/planning |
509373.8523698249 ns (1128.737943117274) |
506096.8931305653 ns (823.0953254230262) |
1.01 |
vortex-pushdown-compressed/exec |
3033073.6629411755 ns (1361.781176470453) |
3058796.5382352932 ns (4489.80602205894) |
0.99 |
vortex-pushdown-uncompressed/planning |
507545.6116696459 ns (688.4101890036545) |
505360.87187101354 ns (832.4561739638157) |
1.00 |
vortex-pushdown-uncompressed/exec |
3341848.059375 ns (15946.811898437561) |
3008000.439411765 ns (6547.197735295165) |
1.11 |
vortex-nopushdown-compressed/planning |
822191.0867838432 ns (760.7616780173266) |
821065.9813587207 ns (659.3040740112774) |
1.00 |
vortex-nopushdown-compressed/exec |
8101823.405714287 ns (7981.783857142553) |
8342180.982857143 ns (51796.87848214293) |
0.97 |
vortex-nopushdown-uncompressed/planning |
813871.629072468 ns (515.3510934513179) |
817852.9513202506 ns (827.0419414441567) |
1.00 |
vortex-nopushdown-uncompressed/exec |
1767535.2524924332 ns (4561.813956841361) |
1768495.8975418287 ns (1570.829725839314) |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Vortex Compression
| Benchmark suite | Current: 601e840 | Previous: 389e6a4 | Ratio |
|---|---|---|---|
Yellow Taxi Trip Data Compression Time/taxi compression |
3027899399.4 ns (2624239.9799997807) |
2381955022 ns (4520086.658750057) |
1.27 |
Yellow Taxi Trip Data Compression Time/taxi compression throughput |
470808924 bytes |
470808924 bytes |
1 |
Yellow Taxi Trip Data Vortex-to-ParquetZstd Ratio/taxi |
0.9525215936770424 ratio |
0.97278717553993 ratio |
0.98 |
Yellow Taxi Trip Data Vortex-to-ParquetUncompressed Ratio/taxi |
0.6114427337828364 ratio |
0.624451617631978 ratio |
0.98 |
Yellow Taxi Trip Data Compression Ratio/taxi |
0.1057036612160733 ratio |
0.10826329833968908 ratio |
0.98 |
Yellow Taxi Trip Data Compression Size/taxi |
49766227 bytes |
50971327 bytes |
0.98 |
Public BI Compression Time/AirlineSentiment compression |
330472.76823708514 ns (195.01859096676344) |
271433.80937275605 ns (303.1445150108775) |
1.22 |
Public BI Compression Time/AirlineSentiment compression throughput |
2020 bytes |
2020 bytes |
1 |
Public BI Vortex-to-ParquetZstd Ratio/AirlineSentiment |
6.666666666666667 ratio |
6.400830737279335 ratio |
1.04 |
Public BI Vortex-to-ParquetUncompressed Ratio/AirlineSentiment |
4.533898305084746 ratio |
4.353107344632768 ratio |
1.04 |
Public BI Compression Ratio/AirlineSentiment |
0.6207920792079208 ratio |
0.6207920792079208 ratio |
1 |
Public BI Compression Size/AirlineSentiment |
1254 bytes |
1254 bytes |
1 |
Public BI Compression Time/Arade compression |
3964777820.9 ns (5853575.522500038) |
3490069468.7 ns (6432250.75) |
1.14 |
Public BI Compression Time/Arade compression throughput |
787023760 bytes |
787023760 bytes |
1 |
Public BI Vortex-to-ParquetZstd Ratio/Arade |
0.47695471917217275 ratio |
0.4797457991144548 ratio |
0.99 |
Public BI Vortex-to-ParquetUncompressed Ratio/Arade |
0.425720664367195 ratio |
0.4282119289664689 ratio |
0.99 |
Public BI Compression Ratio/Arade |
0.17718150211881786 ratio |
0.1784789686654441 ratio |
0.99 |
Public BI Compression Size/Arade |
139446052 bytes |
140467189 bytes |
0.99 |
Public BI Compression Time/Bimbo compression |
28146481739.7 ns (25600846.150001526) |
20498308740.6 ns (14679521.299999237) |
1.37 |
Public BI Compression Time/Bimbo compression throughput |
7121333608 bytes |
7121333608 bytes |
1 |
Public BI Vortex-to-ParquetZstd Ratio/Bimbo |
1.187104957175036 ratio |
1.2857150079834785 ratio |
0.92 |
Public BI Vortex-to-ParquetUncompressed Ratio/Bimbo |
0.8048966306157361 ratio |
0.8717575237160745 ratio |
0.92 |
Public BI Compression Ratio/Bimbo |
0.057484008548641496 ratio |
0.06313256936803795 ratio |
0.91 |
Public BI Compression Size/Bimbo |
409362802 bytes |
449588088 bytes |
0.91 |
Public BI Compression Time/CMSprovider compression |
15814122800.4 ns (19183096.246250153) |
13922053650.8 ns (37447083.207499504) |
1.14 |
Public BI Compression Time/CMSprovider compression throughput |
5149123964 bytes |
5149123964 bytes |
1 |
Public BI Vortex-to-ParquetZstd Ratio/CMSprovider |
1.1102187617110582 ratio |
1.2864902765836628 ratio |
0.86 |
Public BI Vortex-to-ParquetUncompressed Ratio/CMSprovider |
0.716860364183914 ratio |
0.8306776285869021 ratio |
0.86 |
Public BI Compression Ratio/CMSprovider |
0.1592036914883644 ratio |
0.18687751270460576 ratio |
0.85 |
Public BI Compression Size/CMSprovider |
819759543 bytes |
962255479 bytes |
0.85 |
Public BI Compression Time/Euro2016 compression |
1998985890.5 ns (2082981.7262499332) |
1962982184.2 ns (2226448.912500024) |
1.02 |
Public BI Compression Time/Euro2016 compression throughput |
393253221 bytes |
393253221 bytes |
1 |
Public BI Vortex-to-ParquetZstd Ratio/Euro2016 |
1.3767328624200381 ratio |
1.4121892196073749 ratio |
0.97 |
Public BI Vortex-to-ParquetUncompressed Ratio/Euro2016 |
0.5841178925246376 ratio |
0.5991612558394803 ratio |
0.97 |
Public BI Compression Ratio/Euro2016 |
0.4103643438434799 ratio |
0.42094432075865945 ratio |
0.97 |
Public BI Compression Size/Euro2016 |
161377100 bytes |
165537710 bytes |
0.97 |
Public BI Compression Time/Food compression |
1291908123.7 ns (1051553.2362499237) |
1150796322 ns (1520353.861250043) |
1.12 |
Public BI Compression Time/Food compression throughput |
332718229 bytes |
332718229 bytes |
1 |
Public BI Vortex-to-ParquetZstd Ratio/Food |
1.2494585248008683 ratio |
1.270392943510939 ratio |
0.98 |
Public BI Vortex-to-ParquetUncompressed Ratio/Food |
0.7064742936163867 ratio |
0.7183111240328456 ratio |
0.98 |
Public BI Compression Ratio/Food |
0.12975492244520212 ratio |
0.13312083961591417 ratio |
0.97 |
Public BI Compression Size/Food |
43171828 bytes |
44291730 bytes |
0.97 |
Public BI Compression Time/HashTags compression |
3067848353.1 ns (1735290.6662499905) |
2786161120.9 ns (2560013.1875) |
1.10 |
Public BI Compression Time/HashTags compression throughput |
804495592 bytes |
804495592 bytes |
1 |
Public BI Vortex-to-ParquetZstd Ratio/HashTags |
1.5078951114573453 ratio |
1.6137214125109052 ratio |
0.93 |
Public BI Vortex-to-ParquetUncompressed Ratio/HashTags |
0.42869755740814547 ratio |
0.4587841837434173 ratio |
0.93 |
Public BI Compression Ratio/HashTags |
0.24675172987150437 ratio |
0.2648727502288167 ratio |
0.93 |
Public BI Compression Size/HashTags |
198510679 bytes |
213088960 bytes |
0.93 |
TPC-H l_comment Compression Time/chunked-without-fsst compression |
185037147.9266468 ns (1323400.2330314964) |
189276040.32619047 ns (233028.67346726358) |
0.98 |
TPC-H l_comment Compression Time/chunked-without-fsst compression throughput |
183010921 bytes |
183010921 bytes |
1 |
TPC-H l_comment Vortex-to-ParquetZstd Ratio/chunked-without-fsst |
3.2154002617129085 ratio |
3.2154914418075937 ratio |
1.00 |
TPC-H l_comment Vortex-to-ParquetUncompressed Ratio/chunked-without-fsst |
0.998387334976936 ratio |
0.9983753650918578 ratio |
1.00 |
TPC-H l_comment Compression Ratio/chunked-without-fsst |
0.999965750677797 ratio |
0.999965750677797 ratio |
1 |
TPC-H l_comment Compression Size/chunked-without-fsst |
183004653 bytes |
183004653 bytes |
1 |
TPC-H l_comment Compression Time/chunked-with-fsst compression |
1197722390.6 ns (363037.37000000477) |
1181850208.05 ns (3772275.663125038) |
1.01 |
TPC-H l_comment Compression Time/chunked-with-fsst compression throughput |
183010921 bytes |
183010921 bytes |
1 |
TPC-H l_comment Vortex-to-ParquetZstd Ratio/chunked-with-fsst |
1.162075496999417 ratio |
1.3504680080344365 ratio |
0.86 |
TPC-H l_comment Vortex-to-ParquetUncompressed Ratio/chunked-with-fsst |
0.36082644898249266 ratio |
0.4193057313218411 ratio |
0.86 |
TPC-H l_comment Compression Ratio/chunked-with-fsst |
0.35912807083245046 ratio |
0.4181606899841786 ratio |
0.86 |
TPC-H l_comment Compression Size/chunked-with-fsst |
65724359 bytes |
76527973 bytes |
0.86 |
TPC-H l_comment Compression Time/canonical-with-fsst compression |
1194391867.8 ns (255114.18375003338) |
1181414329.4 ns (3882526.791875005) |
1.01 |
TPC-H l_comment Compression Time/canonical-with-fsst compression throughput |
183010937 bytes |
183010937 bytes |
1 |
TPC-H l_comment Vortex-to-ParquetZstd Ratio/canonical-with-fsst |
1.1620679453299145 ratio |
1.3504642840827281 ratio |
0.86 |
TPC-H l_comment Vortex-to-ParquetUncompressed Ratio/canonical-with-fsst |
0.36082650407977923 ratio |
0.4193057473283507 ratio |
0.86 |
TPC-H l_comment Compression Ratio/canonical-with-fsst |
0.35912008362647746 ratio |
0.41815269761719215 ratio |
0.86 |
TPC-H l_comment Compression Size/canonical-with-fsst |
65722903 bytes |
76526517 bytes |
0.86 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
TPC-H
| Benchmark suite | Current: 601e840 | Previous: 389e6a4 | Ratio |
|---|---|---|---|
tpch_q1/vortex-in-memory-no-pushdown |
468637246.6 ns (2043579.699999988) |
467706859.1 ns (2482393.704374999) |
1.00 |
tpch_q1/vortex-in-memory-pushdown |
532035135.3 ns (1121947.3999999762) |
529423161.1 ns (1914717.449999988) |
1.00 |
tpch_q1/arrow |
452415818.2 ns (826248.134375006) |
450505960.1 ns (674118.09375) |
1.00 |
tpch_q1/parquet |
659383201.7 ns (2261171.3562500477) |
653888591.8 ns (1705047.0774999857) |
1.01 |
tpch_q1/vortex-file-compressed |
669875265.8 ns (1803084.7599999905) |
616268578.1 ns (1690019.199999988) |
1.09 |
tpch_q1/vortex-file-uncompressed |
545090685.5 ns (1178417.1499999762) |
546741384.3 ns (3268037.247499943) |
1.00 |
tpch_q2/vortex-in-memory-no-pushdown |
124641736.86694443 ns (446706.6107986048) |
122351660.11047618 ns (146165.53892858326) |
1.02 |
tpch_q2/vortex-in-memory-pushdown |
124935177.74007936 ns (374981.7838670537) |
122353640.38722222 ns (431106.2754374966) |
1.02 |
tpch_q2/arrow |
124837241.0857143 ns (194076.2923809588) |
119793416.16972223 ns (245965.65198262036) |
1.04 |
tpch_q2/parquet |
164140869.18226188 ns (1217396.934870556) |
152737297.16396827 ns (737394.7409236133) |
1.07 |
tpch_q2/vortex-file-compressed |
155482033.86472222 ns (990371.3944097161) |
152132280.3025 ns (382935.9566666633) |
1.02 |
tpch_q2/vortex-file-uncompressed |
159867307.38698414 ns (1367171.452271834) |
154360005.1547619 ns (994986.2169970423) |
1.04 |
tpch_q3/vortex-in-memory-no-pushdown |
162008479.67198414 ns (944365.812171638) |
151890685.88206348 ns (152838.90308234096) |
1.07 |
tpch_q3/vortex-in-memory-pushdown |
194512119.03333333 ns (1642208.0750000179) |
180003441.70749998 ns (642849.3787499964) |
1.08 |
tpch_q3/arrow |
149686423.6454762 ns (798260.0834523886) |
143374803.07503968 ns (328245.98892360926) |
1.04 |
tpch_q3/parquet |
351697489.45 ns (10747557.611249983) |
348836189.8 ns (2598111.5725000203) |
1.01 |
tpch_q3/vortex-file-compressed |
568207008.5 ns (3325726.6087499857) |
328656095.4 ns (685316.3000000119) |
1.73 |
tpch_q3/vortex-file-uncompressed |
283487267.1 ns (1347802.724999994) |
282722805.9 ns (1032179.0512500107) |
1.00 |
tpch_q4/vortex-in-memory-no-pushdown |
112476786.56849205 ns (687449.8152400851) |
109326108.12472221 ns (188698.88503819704) |
1.03 |
tpch_q4/vortex-in-memory-pushdown |
138750666.79329365 ns (2234398.3424464166) |
140895010.61444443 ns (1712144.0433402658) |
0.98 |
tpch_q4/arrow |
104099525.34420635 ns (739040.4402708337) |
103580143.0748016 ns (473460.28302529454) |
1.01 |
tpch_q4/parquet |
224920164 ns (1280517.827916652) |
221881395.06666666 ns (5369896.595833302) |
1.01 |
tpch_q4/vortex-file-compressed |
561755868.9 ns (2917408.269999981) |
287132970.8 ns (845223.7525000274) |
1.96 |
tpch_q4/vortex-file-uncompressed |
224331418.6 ns (963935.8891666383) |
230207994.33333334 ns (1101725.4166666567) |
0.97 |
tpch_q5/vortex-in-memory-no-pushdown |
291936743.2 ns (1220226.75) |
289304603.65 ns (388916.97937500477) |
1.01 |
tpch_q5/vortex-in-memory-pushdown |
297641512.3 ns (1046060.3193749785) |
298672844.5 ns (615663.0225000083) |
1.00 |
tpch_q5/arrow |
278930818.65 ns (785779.9500000477) |
280143002.55 ns (759700.9887500107) |
1.00 |
tpch_q5/parquet |
438604743.85 ns (2037297.4356250167) |
437243441.6 ns (963578.2868750095) |
1.00 |
tpch_q5/vortex-file-compressed |
352280264.45 ns (5655839.946874976) |
357086409.45 ns (4817641.056250006) |
0.99 |
tpch_q5/vortex-file-uncompressed |
342856223 ns (1282297.4631249905) |
352451022.15 ns (2960303.900000006) |
0.97 |
tpch_q6/vortex-in-memory-no-pushdown |
40563522.989034384 ns (181236.8375158757) |
41418057.956269845 ns (268344.0079067424) |
0.98 |
tpch_q6/vortex-in-memory-pushdown |
86407820.67121032 ns (183435.50289410353) |
90060620.66531745 ns (1146554.100840278) |
0.96 |
tpch_q6/arrow |
34742481.27912699 ns (118123.77100595087) |
36745641.538240746 ns (73933.14955092594) |
0.95 |
tpch_q6/parquet |
150007538.7104762 ns (194151.07619048655) |
152311324.27492064 ns (385396.5065972358) |
0.98 |
tpch_q6/vortex-file-compressed |
66007447.07478175 ns (199796.3275873065) |
68321392.75835316 ns (594722.824069947) |
0.97 |
tpch_q6/vortex-file-uncompressed |
174966880.8042857 ns (643927.272380963) |
179440963.3109127 ns (427751.369375512) |
0.98 |
tpch_q7/vortex-in-memory-no-pushdown |
557218729.3 ns (2628618.5) |
561994486.1 ns (1013089.2162500024) |
0.99 |
tpch_q7/vortex-in-memory-pushdown |
600041324.6 ns (1574992.3912500143) |
607182278.5 ns (725857.2599999905) |
0.99 |
tpch_q7/arrow |
546272788.7 ns (5608175.226249993) |
557652690.2 ns (4873358.014999986) |
0.98 |
tpch_q7/parquet |
685403067.4 ns (1960096.1437499523) |
711573978.3 ns (3517538.550000012) |
0.96 |
tpch_q7/vortex-file-compressed |
843016990.2 ns (4618183.54124999) |
721340482.6 ns (2600173.4749999642) |
1.17 |
tpch_q7/vortex-file-uncompressed |
690664554.8 ns (2389644.5) |
698931612.2 ns (4304570.417500019) |
0.99 |
tpch_q8/vortex-in-memory-no-pushdown |
216447234.43333334 ns (209847.021666646) |
214671219.2333333 ns (1720677.7025000006) |
1.01 |
tpch_q8/vortex-in-memory-pushdown |
232054713 ns (1305130.2233333439) |
227132109.56666666 ns (319825.2666666508) |
1.02 |
tpch_q8/arrow |
209454563.26666665 ns (280884.88916666806) |
206622908.6 ns (255661.60916665196) |
1.01 |
tpch_q8/parquet |
473931308.1 ns (2772782.323125005) |
474538273.25 ns (900393.6875) |
1.00 |
tpch_q8/vortex-file-compressed |
294155088.15 ns (958509.0249999762) |
293835397.3 ns (782487.1249999702) |
1.00 |
tpch_q8/vortex-file-uncompressed |
280930962.75 ns (732009.099999994) |
281282068.5 ns (870672.9731250107) |
1.00 |
tpch_q9/vortex-in-memory-no-pushdown |
403960572.9 ns (3106668.1181249917) |
396488574.8 ns (982068.6299999952) |
1.02 |
tpch_q9/vortex-in-memory-pushdown |
417954497.3 ns (1086092.051875025) |
399901425.3 ns (1962670.7587499917) |
1.05 |
tpch_q9/arrow |
387019072.45 ns (768485.9187499881) |
392705014.7 ns (1112274.0937500298) |
0.99 |
tpch_q9/parquet |
678412942.6 ns (987697.8537499905) |
675068552.7 ns (1089853.2012500167) |
1.00 |
tpch_q9/vortex-file-compressed |
496061127.15 ns (845329.5606250167) |
450933879.8 ns (2656224.0068750083) |
1.10 |
tpch_q9/vortex-file-uncompressed |
438815102.4 ns (1227699.75) |
429269320.75 ns (1140982.1787499785) |
1.02 |
tpch_q10/vortex-in-memory-no-pushdown |
223540658.49999994 ns (316261.3979166597) |
223431208.1333333 ns (302109.55916669965) |
1.00 |
tpch_q10/vortex-in-memory-pushdown |
255581068.8 ns (993228) |
258702480.3 ns (1791353.1743749976) |
0.99 |
tpch_q10/arrow |
220750764.03333336 ns (370660.8220833242) |
218960612.8666667 ns (373163.18958333135) |
1.01 |
tpch_q10/parquet |
473401149.1 ns (807295.8362500072) |
476933901.1 ns (2666802.1787500083) |
0.99 |
tpch_q10/vortex-file-compressed |
556456832.1 ns (1348485.5799999833) |
451720548.35 ns (2453389.3243750036) |
1.23 |
tpch_q10/vortex-file-uncompressed |
350401438.25 ns (1052118.474999994) |
359571693.6 ns (836841.3850000203) |
0.97 |
tpch_q11/vortex-in-memory-no-pushdown |
173627874.81746036 ns (218454.79531747103) |
171686489.14805555 ns (163711.0463541597) |
1.01 |
tpch_q11/vortex-in-memory-pushdown |
172252152.1574603 ns (135423.8821131289) |
171469174.08916667 ns (277086.44332292676) |
1.00 |
tpch_q11/arrow |
173383571.8397619 ns (896119.9564762115) |
170484906.82480156 ns (426472.690407753) |
1.02 |
tpch_q11/parquet |
178444794.4835714 ns (273728.8662470281) |
179417656.9555159 ns (869822.7030382156) |
0.99 |
tpch_q11/vortex-file-compressed |
222026351.63333336 ns (678667.3300000131) |
222551598.0333333 ns (718075.0379166752) |
1.00 |
tpch_q11/vortex-file-uncompressed |
221138378.53333336 ns (547325.3162500262) |
222951078.2 ns (870504.9166666716) |
0.99 |
tpch_q12/vortex-in-memory-no-pushdown |
197881998.99999997 ns (101252.38750000298) |
199667226.76666668 ns (502055.45041668415) |
0.99 |
tpch_q12/vortex-in-memory-pushdown |
246435918.8666667 ns (724255.5275000036) |
245538174.06666666 ns (558339.1499999762) |
1.00 |
tpch_q12/arrow |
165710390.0101984 ns (119703.17378619313) |
167913663.72535712 ns (563943.6866741031) |
0.99 |
tpch_q12/parquet |
353457341.65 ns (616018.8524999917) |
356430612.85 ns (717870.2487500012) |
0.99 |
tpch_q12/vortex-file-compressed |
636214405.8 ns (1133189.8675000072) |
640537318.8 ns (1281681.560000062) |
0.99 |
tpch_q12/vortex-file-uncompressed |
351716283.55 ns (736909.1924999952) |
356341509.15 ns (1406323.925000012) |
0.99 |
tpch_q13/vortex-in-memory-no-pushdown |
158678905.12726194 ns (737906.240601182) |
175634501.84150794 ns (2567600.786944449) |
0.90 |
tpch_q13/vortex-in-memory-pushdown |
158437435.9261508 ns (378998.1656577587) |
173174528.8207143 ns (1290509.039970249) |
0.91 |
tpch_q13/arrow |
157003054.45222226 ns (918224.810944438) |
157604793.4111111 ns (1102110.8194791526) |
1.00 |
tpch_q13/parquet |
299064282.35 ns (626718.3549999893) |
302319424.95 ns (1172502.6775000095) |
0.99 |
tpch_q13/vortex-file-compressed |
210731412.03333333 ns (4233877.606250003) |
210353122.6666667 ns (2179664.632499993) |
1.00 |
tpch_q13/vortex-file-uncompressed |
188220076.73333332 ns (873110.7962499857) |
188989524.86666667 ns (642939.3504166603) |
1.00 |
tpch_q14/vortex-in-memory-no-pushdown |
44914342.33079366 ns (166361.23620237783) |
45688479.9459127 ns (323521.9699568413) |
0.98 |
tpch_q14/vortex-in-memory-pushdown |
87635964.26107143 ns (1387757.8773526847) |
82895740.02531746 ns (199121.09422123432) |
1.06 |
tpch_q14/arrow |
37336868.452208996 ns (442046.34431216493) |
38329933.31309524 ns (125707.33668451756) |
0.97 |
tpch_q14/parquet |
221103244.6 ns (352058.93958331645) |
219194986.3 ns (511013.70041665435) |
1.01 |
tpch_q14/vortex-file-compressed |
120170288.28583333 ns (437932.8540208265) |
120736373.63253967 ns (470688.36722221226) |
1.00 |
tpch_q14/vortex-file-uncompressed |
149876196.96976188 ns (300143.39405059814) |
154932860.4807143 ns (437040.4623273909) |
0.97 |
tpch_q15/vortex-in-memory-no-pushdown |
73255224.63045634 ns (351481.9323214218) |
73534529.25388889 ns (168442.9383368045) |
1.00 |
tpch_q15/vortex-in-memory-pushdown |
117184202.56123018 ns (244617.36576735973) |
116857209.02908728 ns (428110.76256696135) |
1.00 |
tpch_q15/arrow |
63459015.50676588 ns (78806.36980579793) |
63581679.31077381 ns (176071.64917782694) |
1.00 |
tpch_q15/parquet |
290448086.25 ns (638761.6062500179) |
290358343.8 ns (446317.22687500715) |
1.00 |
tpch_q15/vortex-file-compressed |
219363046.0333333 ns (722322.0487500131) |
217957832 ns (805120.3283333182) |
1.01 |
tpch_q15/vortex-file-uncompressed |
302034883.55 ns (13073341.232499987) |
303672058.75 ns (660240.3806250095) |
0.99 |
tpch_q16/vortex-in-memory-no-pushdown |
105262270.0922619 ns (139785.97434672713) |
105347126.38305555 ns (160069.32227083296) |
1.00 |
tpch_q16/vortex-in-memory-pushdown |
123788781.8406746 ns (197083.78150445968) |
124305309.05678573 ns (173901.09089583158) |
1.00 |
tpch_q16/arrow |
104668873.87043652 ns (469857.93408483267) |
105267876.45773807 ns (344850.4093452394) |
0.99 |
tpch_q16/parquet |
120823613.19055554 ns (362746.5805416778) |
123178301.87309524 ns (745863.5998809561) |
0.98 |
tpch_q16/vortex-file-compressed |
136251009.92825395 ns (143045.60722222924) |
137310037.81515872 ns (473825.8004126996) |
0.99 |
tpch_q16/vortex-file-uncompressed |
130864632.44809525 ns (264562.11390774697) |
133187943.51825395 ns (542618.2346825451) |
0.98 |
tpch_q17/vortex-in-memory-no-pushdown |
542411458.1 ns (3610175.311250031) |
561745569.3 ns (3547320.800000012) |
0.97 |
tpch_q17/vortex-in-memory-pushdown |
625668305.2 ns (4656901.277500033) |
631154291.3 ns (18076098.949999988) |
0.99 |
tpch_q17/arrow |
535046087.4 ns (5028765.871250033) |
542748157 ns (2734965.649999976) |
0.99 |
tpch_q17/parquet |
585949251.9 ns (1365669.8274999857) |
582818562.3 ns (1254511.0099999905) |
1.01 |
tpch_q17/vortex-file-compressed |
717365283.4 ns (8856988.743750036) |
616318929.1 ns (1178975.4624999762) |
1.16 |
tpch_q17/vortex-file-uncompressed |
637579582.1 ns (7395510.237499952) |
601636065.6 ns (1452584) |
1.06 |
tpch_q18/vortex-in-memory-no-pushdown |
1036619576.2 ns (8129133.017499983) |
1005002198.2 ns (6979879.800000012) |
1.03 |
tpch_q18/vortex-in-memory-pushdown |
1054220798 ns (13516203.875) |
1001500025.9 ns (7364370.449999988) |
1.05 |
tpch_q18/arrow |
1019376576.3 ns (8325333.118750036) |
1008280714.3 ns (6935897.1450000405) |
1.01 |
tpch_q18/parquet |
1186580173.9 ns (5202922.117499948) |
1170119606.5 ns (9346595.472499967) |
1.01 |
tpch_q18/vortex-file-compressed |
1151950918.7 ns (6281022.936249971) |
1044123594.8 ns (2992860.5) |
1.10 |
tpch_q18/vortex-file-uncompressed |
1074494049.1 ns (14093675.143749952) |
1030449932.5 ns (4612716.768750012) |
1.04 |
tpch_q19/vortex-in-memory-no-pushdown |
160790367.9642857 ns (284240.70937500894) |
161341363.59531745 ns (510302.61159424484) |
1.00 |
tpch_q19/vortex-in-memory-pushdown |
240596218.86666664 ns (1273749.581666693) |
241483164.6333333 ns (843957.4229166657) |
1.00 |
tpch_q19/arrow |
151049751.65361112 ns (363945.7744618207) |
151202048.85579365 ns (253001.50577183068) |
1.00 |
tpch_q19/parquet |
470577447.75 ns (463987.08812499046) |
475896366.55 ns (1195324.7349999845) |
0.99 |
tpch_q19/vortex-file-compressed |
982842022.9 ns (5515158.857500017) |
613593120.8 ns (948408.6999999881) |
1.60 |
tpch_q19/vortex-file-uncompressed |
316772785.7 ns (1064581.699999988) |
318309993.15 ns (752943.9662499726) |
1.00 |
tpch_q20/vortex-in-memory-no-pushdown |
246298305.7333333 ns (994434.5491666943) |
252146000.95 ns (1317447.112499997) |
0.98 |
tpch_q20/vortex-in-memory-pushdown |
266490327.75 ns (1053370.7368749976) |
270465410.2 ns (1103824.065625012) |
0.99 |
tpch_q20/arrow |
237819196.1 ns (475858.3533333391) |
242576892.1 ns (980416.171666652) |
0.98 |
tpch_q20/parquet |
353909770.65 ns (707487.3249999881) |
358184901.9 ns (1024666.2687500119) |
0.99 |
tpch_q20/vortex-file-compressed |
398823814.95 ns (974137.8356249928) |
357019397.65 ns (1089999.5149999857) |
1.12 |
tpch_q20/vortex-file-uncompressed |
375905138.65 ns (686944.1568749845) |
401661314 ns (1680046.892500013) |
0.94 |
tpch_q21/vortex-in-memory-no-pushdown |
832212692.5 ns (1437880.1112499833) |
858036595.9 ns (3027155.032499969) |
0.97 |
tpch_q21/vortex-in-memory-pushdown |
875606578.1 ns (1462515.6424999833) |
915000617.2 ns (6874021.561249971) |
0.96 |
tpch_q21/arrow |
808448543.2 ns (1413725.676249981) |
825291478.2 ns (3510404.519999981) |
0.98 |
tpch_q21/parquet |
951960857.6 ns (1541503.6775000095) |
959471433.4 ns (2789190.835000038) |
0.99 |
tpch_q21/vortex-file-compressed |
1763962353.4 ns (30836298.29250002) |
1232375230.4 ns (2867111.3087500334) |
1.43 |
tpch_q21/vortex-file-uncompressed |
1095468370.6 ns (11372111.826249957) |
1086880916.1 ns (4277297.965000033) |
1.01 |
tpch_q22/vortex-in-memory-no-pushdown |
67661632.03646825 ns (269610.52954365313) |
67645648.4909127 ns (338350.0951587334) |
1.00 |
tpch_q22/vortex-in-memory-pushdown |
67716248.80486111 ns (277712.35093055665) |
68556594.51313493 ns (248686.2916205302) |
0.99 |
tpch_q22/arrow |
65312735.19988096 ns (162259.00596280023) |
66241165.37230159 ns (169823.53043550625) |
0.99 |
tpch_q22/parquet |
93724444.25892857 ns (246818.5087499991) |
94547036.87111111 ns (290610.0577777773) |
0.99 |
tpch_q22/vortex-file-compressed |
102042275.2193254 ns (528112.4244345203) |
101214570.9536111 ns (332681.64763888717) |
1.01 |
tpch_q22/vortex-file-uncompressed |
101350147.8975 ns (284561.421322912) |
100769892.65698412 ns (242336.10545933247) |
1.01 |
This comment was automatically generated by workflow using github-action-benchmark.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We fairly often perform FoR and then do not perform bitpacking because we have exceeded the depth. FoR is very cheap and only worthwhile in combination with bitpacking anyway, so setting its cost to zero should improve compression ratio with a minor decrease in compresssion speed.
I've been poking around trying to make the compressor reliably faster and along the way I noticed that this noticeably improves compression ratio with a minor impact on compression speed.