-
Notifications
You must be signed in to change notification settings - Fork 2
/
06b-full.txt
275 lines (253 loc) · 8.88 KB
/
06b-full.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
# JMH version: 1.36
# VM version: JDK 17.0.7, OpenJDK 64-Bit Server VM, 17.0.7+7
# VM invoker: /home/r_schwietzke/.sdkman/candidates/java/17.0.7-tem/bin/java
# VM options: -Xms2g -Xmx2g -XX:+UseSerialGC -XX:+AlwaysPreTouch -XX:+UseSerialGC
# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 3 iterations, 2 s each
# Measurement: 10 iterations, 2 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.xceptance.B06b_QuotedWarmupAndUnquotedTest.parse
# Run progress: 0.00% complete, ETA 00:06:30
# Fork: 1 of 15
# Warmup Iteration 1: 676.655 ns/op
# Warmup Iteration 2: 717.067 ns/op
# Warmup Iteration 3: 709.536 ns/op
Iteration 1: 1004.059 ns/op
Iteration 2: 1003.092 ns/op
Iteration 3: 1001.875 ns/op
Iteration 4: 1001.421 ns/op
Iteration 5: 1000.465 ns/op
Iteration 6: 1000.704 ns/op
Iteration 7: 1006.352 ns/op
Iteration 8: 999.966 ns/op
Iteration 9: 999.657 ns/op
Iteration 10: 1000.144 ns/op
# Run progress: 6.67% complete, ETA 00:06:20
# Fork: 2 of 15
# Warmup Iteration 1: 680.646 ns/op
# Warmup Iteration 2: 677.395 ns/op
# Warmup Iteration 3: 670.860 ns/op
Iteration 1: 1071.145 ns/op
Iteration 2: 1070.235 ns/op
Iteration 3: 1068.818 ns/op
Iteration 4: 1068.299 ns/op
Iteration 5: 1070.247 ns/op
Iteration 6: 1068.106 ns/op
Iteration 7: 1068.930 ns/op
Iteration 8: 1068.248 ns/op
Iteration 9: 1067.400 ns/op
Iteration 10: 1068.310 ns/op
# Run progress: 13.33% complete, ETA 00:05:53
# Fork: 3 of 15
# Warmup Iteration 1: 686.646 ns/op
# Warmup Iteration 2: 715.839 ns/op
# Warmup Iteration 3: 707.962 ns/op
Iteration 1: 1004.361 ns/op
Iteration 2: 1002.522 ns/op
Iteration 3: 1000.865 ns/op
Iteration 4: 1000.935 ns/op
Iteration 5: 1008.903 ns/op
Iteration 6: 1001.248 ns/op
Iteration 7: 1001.265 ns/op
Iteration 8: 999.919 ns/op
Iteration 9: 999.308 ns/op
Iteration 10: 999.807 ns/op
# Run progress: 20.00% complete, ETA 00:05:26
# Fork: 4 of 15
# Warmup Iteration 1: 680.473 ns/op
# Warmup Iteration 2: 715.465 ns/op
# Warmup Iteration 3: 707.656 ns/op
Iteration 1: 1004.823 ns/op
Iteration 2: 1001.995 ns/op
Iteration 3: 1000.989 ns/op
Iteration 4: 999.706 ns/op
Iteration 5: 1000.068 ns/op
Iteration 6: 1001.395 ns/op
Iteration 7: 1000.453 ns/op
Iteration 8: 1001.717 ns/op
Iteration 9: 1000.412 ns/op
Iteration 10: 1000.246 ns/op
# Run progress: 26.67% complete, ETA 00:04:59
# Fork: 5 of 15
# Warmup Iteration 1: 677.743 ns/op
# Warmup Iteration 2: 677.943 ns/op
# Warmup Iteration 3: 670.710 ns/op
Iteration 1: 1069.916 ns/op
Iteration 2: 1071.179 ns/op
Iteration 3: 1069.353 ns/op
Iteration 4: 1068.533 ns/op
Iteration 5: 1069.363 ns/op
Iteration 6: 1072.679 ns/op
Iteration 7: 1068.734 ns/op
Iteration 8: 1070.396 ns/op
Iteration 9: 1068.960 ns/op
Iteration 10: 1068.218 ns/op
# Run progress: 33.33% complete, ETA 00:04:31
# Fork: 6 of 15
# Warmup Iteration 1: 677.324 ns/op
# Warmup Iteration 2: 678.839 ns/op
# Warmup Iteration 3: 671.118 ns/op
Iteration 1: 1073.296 ns/op
Iteration 2: 1070.203 ns/op
Iteration 3: 1070.355 ns/op
Iteration 4: 1068.431 ns/op
Iteration 5: 1067.759 ns/op
Iteration 6: 1068.256 ns/op
Iteration 7: 1069.083 ns/op
Iteration 8: 1069.325 ns/op
Iteration 9: 1068.554 ns/op
Iteration 10: 1068.361 ns/op
# Run progress: 40.00% complete, ETA 00:04:04
# Fork: 7 of 15
# Warmup Iteration 1: 623.597 ns/op
# Warmup Iteration 2: 714.708 ns/op
# Warmup Iteration 3: 707.871 ns/op
Iteration 1: 1002.635 ns/op
Iteration 2: 1001.878 ns/op
Iteration 3: 1001.018 ns/op
Iteration 4: 1000.323 ns/op
Iteration 5: 1000.613 ns/op
Iteration 6: 1000.497 ns/op
Iteration 7: 1015.572 ns/op
Iteration 8: 999.853 ns/op
Iteration 9: 999.108 ns/op
Iteration 10: 999.838 ns/op
# Run progress: 46.67% complete, ETA 00:03:37
# Fork: 8 of 15
# Warmup Iteration 1: 680.624 ns/op
# Warmup Iteration 2: 678.179 ns/op
# Warmup Iteration 3: 670.244 ns/op
Iteration 1: 1070.980 ns/op
Iteration 2: 1069.254 ns/op
Iteration 3: 1069.481 ns/op
Iteration 4: 1070.024 ns/op
Iteration 5: 1068.006 ns/op
Iteration 6: 1069.947 ns/op
Iteration 7: 1070.091 ns/op
Iteration 8: 1067.933 ns/op
Iteration 9: 1068.169 ns/op
Iteration 10: 1068.110 ns/op
# Run progress: 53.33% complete, ETA 00:03:10
# Fork: 9 of 15
# Warmup Iteration 1: 679.477 ns/op
# Warmup Iteration 2: 714.960 ns/op
# Warmup Iteration 3: 707.683 ns/op
Iteration 1: 1004.461 ns/op
Iteration 2: 1001.764 ns/op
Iteration 3: 1000.861 ns/op
Iteration 4: 999.964 ns/op
Iteration 5: 1000.498 ns/op
Iteration 6: 1000.181 ns/op
Iteration 7: 1000.310 ns/op
Iteration 8: 1000.301 ns/op
Iteration 9: 999.355 ns/op
Iteration 10: 999.538 ns/op
# Run progress: 60.00% complete, ETA 00:02:43
# Fork: 10 of 15
# Warmup Iteration 1: 680.105 ns/op
# Warmup Iteration 2: 715.123 ns/op
# Warmup Iteration 3: 707.908 ns/op
Iteration 1: 1003.016 ns/op
Iteration 2: 1001.244 ns/op
Iteration 3: 1000.733 ns/op
Iteration 4: 1001.540 ns/op
Iteration 5: 1000.317 ns/op
Iteration 6: 999.840 ns/op
Iteration 7: 1000.947 ns/op
Iteration 8: 999.754 ns/op
Iteration 9: 999.968 ns/op
Iteration 10: 1000.217 ns/op
# Run progress: 66.67% complete, ETA 00:02:15
# Fork: 11 of 15
# Warmup Iteration 1: 677.398 ns/op
# Warmup Iteration 2: 715.767 ns/op
# Warmup Iteration 3: 707.589 ns/op
Iteration 1: 1003.023 ns/op
Iteration 2: 1000.862 ns/op
Iteration 3: 1001.655 ns/op
Iteration 4: 1000.358 ns/op
Iteration 5: 999.606 ns/op
Iteration 6: 1000.092 ns/op
Iteration 7: 999.606 ns/op
Iteration 8: 999.542 ns/op
Iteration 9: 999.402 ns/op
Iteration 10: 999.988 ns/op
# Run progress: 73.33% complete, ETA 00:01:48
# Fork: 12 of 15
# Warmup Iteration 1: 679.279 ns/op
# Warmup Iteration 2: 678.607 ns/op
# Warmup Iteration 3: 670.577 ns/op
Iteration 1: 1070.464 ns/op
Iteration 2: 1071.340 ns/op
Iteration 3: 1071.273 ns/op
Iteration 4: 1068.385 ns/op
Iteration 5: 1067.723 ns/op
Iteration 6: 1067.855 ns/op
Iteration 7: 1068.598 ns/op
Iteration 8: 1067.758 ns/op
Iteration 9: 1067.492 ns/op
Iteration 10: 1068.063 ns/op
# Run progress: 80.00% complete, ETA 00:01:21
# Fork: 13 of 15
# Warmup Iteration 1: 677.039 ns/op
# Warmup Iteration 2: 677.258 ns/op
# Warmup Iteration 3: 670.924 ns/op
Iteration 1: 1071.900 ns/op
Iteration 2: 1071.926 ns/op
Iteration 3: 1071.679 ns/op
Iteration 4: 1070.878 ns/op
Iteration 5: 1068.647 ns/op
Iteration 6: 1068.793 ns/op
Iteration 7: 1068.889 ns/op
Iteration 8: 1071.048 ns/op
Iteration 9: 1073.313 ns/op
Iteration 10: 1070.768 ns/op
# Run progress: 86.67% complete, ETA 00:00:54
# Fork: 14 of 15
# Warmup Iteration 1: 677.130 ns/op
# Warmup Iteration 2: 678.200 ns/op
# Warmup Iteration 3: 670.647 ns/op
Iteration 1: 1072.349 ns/op
Iteration 2: 1076.597 ns/op
Iteration 3: 1070.518 ns/op
Iteration 4: 1070.051 ns/op
Iteration 5: 1070.515 ns/op
Iteration 6: 1070.336 ns/op
Iteration 7: 1071.902 ns/op
Iteration 8: 1089.472 ns/op
Iteration 9: 1069.365 ns/op
Iteration 10: 1069.429 ns/op
# Run progress: 93.33% complete, ETA 00:00:27
# Fork: 15 of 15
# Warmup Iteration 1: 677.147 ns/op
# Warmup Iteration 2: 714.794 ns/op
# Warmup Iteration 3: 707.964 ns/op
Iteration 1: 1002.860 ns/op
Iteration 2: 1001.867 ns/op
Iteration 3: 1000.881 ns/op
Iteration 4: 1001.435 ns/op
Iteration 5: 1000.349 ns/op
Iteration 6: 1000.016 ns/op
Iteration 7: 1000.470 ns/op
Iteration 8: 999.753 ns/op
Iteration 9: 1000.569 ns/op
Iteration 10: 1000.753 ns/op
Result "org.xceptance.B06b_QuotedWarmupAndUnquotedTest.parse":
1033.319 ±(99.9%) 9.463 ns/op [Average]
(min, avg, max) = (999.108, 1033.319, 1089.472), stdev = 34.524
CI (99.9%): [1023.856, 1042.782] (assumes normal distribution)
# Run complete. Total time: 00:06:47
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
NOTE: Current JVM experimentally supports Compiler Blackholes, and they are in use. Please exercise
extra caution when trusting the results, look into the generated code to check the benchmark still
works, and factor in a small probability of new VM bugs. Additionally, while comparisons between
different JVMs are already problematic, the performance difference caused by different Blackhole
modes can be very significant. Please make sure you use the consistent Blackhole mode for comparisons.
Benchmark Mode Cnt Score Error Units
B06b_QuotedWarmupAndUnquotedTest.parse avgt 150 1033.319 ± 9.463 ns/op