Skip to content

Commit

Permalink
Add RunBenchmarkWithCodegen
Browse files Browse the repository at this point in the history
  • Loading branch information
@wangyum
wangyum committed Sep 22, 2018
1 parent e0c6d09 commit 42230b6
Show file tree
Hide file tree
Showing 3 changed files with 149 additions and 116 deletions.
76 changes: 38 additions & 38 deletions sql/core/benchmarks/AggregateBenchmark-results.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

agg w/o group: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
agg w/o group wholestage off 39650 / 46049 52.9 18.9 1.0X
agg w/o group wholestage on 1224 / 1413 1713.5 0.6 32.4X
agg w/o group wholestage off 40454 / 44424 51.8 19.3 1.0X
agg w/o group wholestage on 907 / 929 2312.8 0.4 44.6X


================================================================================================
Expand All @@ -20,16 +20,16 @@ Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

stddev: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
stddev wholestage off 6149 / 6366 17.1 58.6 1.0X
stddev wholestage on 871 / 881 120.4 8.3 7.1X
stddev wholestage off 6281 / 6626 16.7 59.9 1.0X
stddev wholestage on 909 / 987 115.4 8.7 6.9X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

kurtosis: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
kurtosis wholestage off 28822 / 29231 3.6 274.9 1.0X
kurtosis wholestage on 929 / 944 112.9 8.9 31.0X
kurtosis wholestage off 32819 / 33789 3.2 313.0 1.0X
kurtosis wholestage on 988 / 1046 106.1 9.4 33.2X


================================================================================================
Expand All @@ -41,9 +41,9 @@ Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Aggregate w keys: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
codegen = F 7956 / 7967 10.5 94.8 1.0X
codegen = T hashmap = F 3872 / 4049 21.7 46.2 2.1X
codegen = T hashmap = T 872 / 883 96.3 10.4 9.1X
codegen = F 8490 / 8568 9.9 101.2 1.0X
codegen = T hashmap = F 5985 / 6916 14.0 71.4 1.4X
codegen = T hashmap = T 943 / 977 88.9 11.2 9.0X


================================================================================================
Expand All @@ -55,9 +55,9 @@ Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Aggregate w keys: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
codegen = F 9088 / 9240 9.2 108.3 1.0X
codegen = T hashmap = F 5065 / 5238 16.6 60.4 1.8X
codegen = T hashmap = T 1722 / 1768 48.7 20.5 5.3X
codegen = F 9509 / 9893 8.8 113.4 1.0X
codegen = T hashmap = F 5381 / 5715 15.6 64.1 1.8X
codegen = T hashmap = T 1792 / 1874 46.8 21.4 5.3X


================================================================================================
Expand All @@ -69,9 +69,9 @@ Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Aggregate w string key: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
codegen = F 3666 / 3704 5.7 174.8 1.0X
codegen = T hashmap = F 2322 / 2357 9.0 110.7 1.6X
codegen = T hashmap = T 1643 / 1676 12.8 78.3 2.2X
codegen = F 4372 / 4440 4.8 208.5 1.0X
codegen = T hashmap = F 2850 / 2940 7.4 135.9 1.5X
codegen = T hashmap = T 2033 / 2175 10.3 96.9 2.2X


================================================================================================
Expand All @@ -83,9 +83,9 @@ Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Aggregate w decimal key: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
codegen = F 2688 / 2704 7.8 128.2 1.0X
codegen = T hashmap = F 1401 / 1430 15.0 66.8 1.9X
codegen = T hashmap = T 394 / 415 53.2 18.8 6.8X
codegen = F 3819 / 4047 5.5 182.1 1.0X
codegen = T hashmap = F 2080 / 2122 10.1 99.2 1.8X
codegen = T hashmap = T 500 / 523 41.9 23.9 7.6X


================================================================================================
Expand All @@ -97,9 +97,9 @@ Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Aggregate w multiple keys: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
codegen = F 5380 / 5437 3.9 256.5 1.0X
codegen = T hashmap = F 3554 / 3648 5.9 169.5 1.5X
codegen = T hashmap = T 2687 / 2719 7.8 128.1 2.0X
codegen = F 5815 / 6015 3.6 277.3 1.0X
codegen = T hashmap = F 3934 / 3955 5.3 187.6 1.5X
codegen = T hashmap = T 3196 / 3307 6.6 152.4 1.8X


================================================================================================
Expand All @@ -111,9 +111,9 @@ Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

max function bytecode size: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
codegen = F 375 / 416 1.7 572.0 1.0X
codegen = T hugeMethodLimit = 10000 231 / 245 2.8 352.0 1.6X
codegen = T hugeMethodLimit = 1500 383 / 412 1.7 583.7 1.0X
codegen = F 415 / 511 1.6 632.8 1.0X
codegen = T hugeMethodLimit = 10000 246 / 281 2.7 375.5 1.7X
codegen = T hugeMethodLimit = 1500 416 / 472 1.6 635.2 1.0X


================================================================================================
Expand All @@ -125,8 +125,8 @@ Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

cube: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
cube wholestage off 2250 / 2266 2.3 429.1 1.0X
cube wholestage on 907 / 945 5.8 173.0 2.5X
cube wholestage off 2192 / 2270 2.4 418.1 1.0X
cube wholestage on 910 / 1001 5.8 173.5 2.4X


================================================================================================
Expand All @@ -138,17 +138,17 @@ Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

BytesToBytesMap: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
UnsafeRowhash 205 / 215 102.4 9.8 1.0X
murmur3 hash 104 / 111 202.0 4.9 2.0X
fast hash 55 / 60 381.2 2.6 3.7X
arrayEqual 132 / 139 158.9 6.3 1.6X
Java HashMap (Long) 89 / 103 235.9 4.2 2.3X
Java HashMap (two ints) 91 / 107 229.2 4.4 2.2X
Java HashMap (UnsafeRow) 759 / 772 27.6 36.2 0.3X
LongToUnsafeRowMap (opt=false) 384 / 406 54.7 18.3 0.5X
LongToUnsafeRowMap (opt=true) 82 / 88 256.5 3.9 2.5X
BytesToBytesMap (off Heap) 753 / 811 27.8 35.9 0.3X
BytesToBytesMap (on Heap) 765 / 784 27.4 36.5 0.3X
Aggregate HashMap 35 / 39 591.4 1.7 5.8X
UnsafeRowhash 218 / 225 96.4 10.4 1.0X
murmur3 hash 114 / 121 183.3 5.5 1.9X
fast hash 59 / 64 356.3 2.8 3.7X
arrayEqual 143 / 151 146.6 6.8 1.5X
Java HashMap (Long) 99 / 127 211.0 4.7 2.2X
Java HashMap (two ints) 101 / 136 207.2 4.8 2.1X
Java HashMap (UnsafeRow) 781 / 938 26.9 37.2 0.3X
LongToUnsafeRowMap (opt=false) 398 / 412 52.7 19.0 0.5X
LongToUnsafeRowMap (opt=true) 88 / 95 238.1 4.2 2.5X
BytesToBytesMap (off Heap) 1091 / 1113 19.2 52.0 0.2X
BytesToBytesMap (on Heap) 792 / 835 26.5 37.8 0.3X
Aggregate HashMap 36 / 41 587.9 1.7 6.1X


Loading

0 comments on commit 42230b6

Please sign in to comment.