-
Notifications
You must be signed in to change notification settings - Fork 50
Benchmark
Patrick Favre-Bulle edited this page Feb 11, 2023
·
31 revisions
The timing values are for a single bcrypt hash in milliseconds. Until cost 11, 250 rounds are calculated per implementation, from there on up the rounds are halved, beginning with cost 12 using 200, cost 14 50 rounds, etc.
- Hardware: i7-7700K, 24GB Ram
- Software: Java 1.8.0_172 (Oracle Corporation), Windows 10 (10.0)
- Tool-Version: 0.4.0 (250 rounds max)
cost 6 | cost 8 | cost 9 | cost 10 | cost 11 | cost 12 | cost 14 | cost 15 | |
---|---|---|---|---|---|---|---|---|
FavreBcrypt | 3.39 ms | 13.57 ms | 27.19 ms | 55.08 ms | 110.14 ms | 217.71 ms | 865.1 ms | 1733.96 ms |
JBcrypt | 3.42 ms | 13.8 ms | 27.78 ms | 56.31 ms | 110.65 ms | 221.6 ms | 881.0 ms | 1759.64 ms |
BC | 3.12 ms | 12.51 ms | 26.0 ms | 50.59 ms | 100.82 ms | 203.43 ms | 799.86 ms | 1601.0 ms |
- Hardware: i5-6440HQ, 32GB Ram
- Software: Java 1.8.0_172 (Oracle Corporation), Windows 10 (10.0)
- Tool-Version: 0.4.0 (250 rounds max)
cost 6 | cost 8 | cost 9 | cost 10 | cost 11 | cost 12 | cost 14 | cost 15 | |
---|---|---|---|---|---|---|---|---|
FavreBcrypt | 5.74 ms | 21.56 ms | 43.78 ms | 78.54 ms | 156.92 ms | 311.9 ms | 1250.94 ms | 2529.56 ms |
JBcrypt | 5.63 ms | 48.62 ms | 40.54 ms | 80.44 ms | 159.84 ms | 320.58 ms | 1288.32 ms | 2563.68 ms |
BC | 5.13 ms | 24.22 ms | 37.14 ms | 73.43 ms | 147.87 ms | 293.07 ms | 1178.74 ms | 2347.36 ms |
- Hardware: i5-6300U, 8GB Ram
- Software: Java 1.8.0_172 (Oracle Corporation), Windows 10 (10.0)
- Tool-Version: 0.4.0
cost 6 | cost 8 | cost 9 | cost 10 | cost 11 | cost 12 | cost 14 | cost 15 | |
---|---|---|---|---|---|---|---|---|
FavreBcrypt | 6.26 ms | 22.99 ms | 41.97 ms | 95.91 ms | 210.6 ms | 354.71 ms | 1413.58 ms | 2715.72 ms |
JBcrypt | 6.18 ms | 21.58 ms | 42.64 ms | 100.27 ms | 187.34 ms | 371.55 ms | 1385.62 ms | 2802.0 ms |
BC | 5.06 ms | 19.38 ms | 38.87 ms | 97.2 ms | 160.03 ms | 322.63 ms | 1257.66 ms | 2496.52 ms |
- Hardware: M1 Pro, 32G Ram
- Software: Java temurin-1.8.0_362, MacOS 13.2
- Tool-Version: 0.10.1
cost 4 | cost 6 | cost 8 | cost 9 | cost 10 | cost 11 | cost 12 | cost 14 | cost 15 | |
---|---|---|---|---|---|---|---|---|---|
JBcrypt | 1.53 ms | 5.6 ms | 21.59 ms | 41.66 ms | 83.48 ms | 170.68 ms | 331.73 ms | 1326.18 ms | 2648.52 ms |
BC | 1.38 ms | 5.3 ms | 20.6 ms | 40.32 ms | 78.92 ms | 157.67 ms | 317.94 ms | 1260.5 ms | 2521.36 ms |
FavreBcrypt | 1.63 ms | 5.99 ms | 21.9 ms | 41.17 ms | 82.14 ms | 164.42 ms | 328.5 ms | 1312.58 ms | 2621.8 ms |
- Hardware: SM-N950F
- Software: SDK 25, N950FXXS3BRB2, NMF26X, Linux (4.4.13-12472670)
- Tool-Version: 0.4.0 (250 rounds max)
cost 6 | cost 8 | cost 9 | cost 10 | cost 11 | cost 12 | cost 14 | cost 15 | |
---|---|---|---|---|---|---|---|---|
FavreBcrypt | 7.7 ms | 28.67 ms | 56.69 ms | 112.75 ms | 224.94 ms | 449.4 ms | 1796.44 ms | 3594.04 ms |
JBcrypt | 7.71 ms | 29.16 ms | 57.78 ms | 115.0 ms | 229.52 ms | 458.34 ms | 1834.66 ms | 3670.52 ms |
BC | 9.3 ms | 35.57 ms | 70.58 ms | 140.66 ms | 280.91 ms | 560.98 ms | 2245.72 ms | 4504.68 ms |
- Hardware: SM-G955F, 64GB
- Software: SDK 26, G955FXXU2CRF7, R16NW
- Tool-Version: 0.4.0 (250 rounds max)
cost 6 | cost 8 | cost 9 | cost 10 | cost 11 | cost 12 | cost 14 | cost 15 | |
---|---|---|---|---|---|---|---|---|
FavreBcrypt | 8.04 ms | 28.52 ms | 55.72 ms | 110.18 ms | 219.75 ms | 438.69 ms | 1760.12 ms | 3538.72 ms |
JBcrypt | 7.92 ms | 29.43 ms | 58.01 ms | 115.51 ms | 231.14 ms | 462.24 ms | 1856.24 ms | 4037.4 ms |
BC | 10.06 ms | 37.38 ms | 76.74 ms | 152.87 ms | 300.66 ms | 600.19 ms | 2427.3 ms | 4793.96 ms |
- Hardware: walleye
- Software: Android P DP4, PP4.180612.004, Linux (4.4.116-g5ec1df259c0f)
- Tool-Version: 0.4.0 (250 rounds max)
cost 6 | cost 8 | cost 9 | cost 10 | cost 11 | cost 12 | cost 14 | cost 15 | |
---|---|---|---|---|---|---|---|---|
FavreBcrypt | 10.64 ms | 39.33 ms | 77.4 ms | 154.21 ms | 306.48 ms | 611.13 ms | 2442.48 ms | 4877.28 ms |
JBcrypt | 10.65 ms | 39.7 ms | 78.66 ms | 156.1 ms | 311.5 ms | 621.34 ms | 2481.62 ms | 4963.88 ms |
BC | 14.65 ms | 57.34 ms | 114.48 ms | 228.04 ms | 454.45 ms | 908.46 ms | 3632.18 ms | 7267.24 ms |
- Hardware: sailfish
- Software: SDK 27, OPM1.171019.012, Linux (3.18.70-g520f6eb)
- Tool-Version: 0.4.0 (250 rounds max)
cost 6 | cost 8 | cost 9 | cost 10 | cost 11 | cost 12 | cost 14 | cost 15 | |
---|---|---|---|---|---|---|---|---|
FavreBcrypt | 11.74 ms | 50.94 ms | 71.87 ms | 140.08 ms | 276.31 ms | 547.92 ms | 2190.28 ms | 4366.56 ms |
JBcrypt | 11.36 ms | 45.98 ms | 73.52 ms | 160.7 ms | 284.31 ms | 565.24 ms | 2258.44 ms | 4507.68 ms |
BC | 15.49 ms | 89.74 ms | 112.17 ms | 221.06 ms | 439.98 ms | 877.18 ms | 3510.62 ms | 7026.96 ms |
- Hardware: LG-H850
- Software: SDK 24, 18082154383aa, Linux (3.10.73-gd5f156e7af96)
- Tool-Version: 0.4.0 (250 rounds max)
cost 6 | cost 8 | cost 9 | cost 10 | cost 11 | cost 12 | cost 14 | cost 15 | |
---|---|---|---|---|---|---|---|---|
FavreBcrypt | 7.41 ms | 27.4 ms | 56.74 ms | 117.75 ms | 241.93 ms | 490.57 ms | 1986.48 ms | 4005.96 ms |
JBcrypt | 7.24 ms | 28.65 ms | 56.89 ms | 121.64 ms | 246.13 ms | 501.2 ms | 2019.62 ms | 4077.52 ms |
BC | 9.83 ms | 40.52 ms | 82.1 ms | 169.95 ms | 343.74 ms | 703.77 ms | 2841.28 ms | 5694.52 ms |
- Hardware: bullhead
- Software: SDK 27, OPM2.171018.029, Linux (3.18.31-perf-g25cc4c2-00017-gf51f86fa)
- Tool-Version: 0.4.0 (250 rounds max)
cost 6 | cost 8 | cost 9 | cost 10 | cost 11 | cost 12 | cost 14 | cost 15 | |
---|---|---|---|---|---|---|---|---|
FavreBcrypt | 31.03 ms | 104.66 ms | 77.34 ms | 186.39 ms | 810.8 ms | 1617.83 ms | 3860.78 ms | 8216.64 ms |
JBcrypt | 28.02 ms | 39.69 ms | 149.22 ms | 265.72 ms | 841.09 ms | 1175.68 ms | 3763.7 ms | 7939.96 ms |
BC | 50.06 ms | 128.4 ms | 202.93 ms | 408.63 ms | 1540.98 ms | 3086.58 ms | 6670.66 ms | 13826.56 ms |
- Hardware: cedric
- Software: SDK 24, NPPS25.137-72-4
- Tool-Version: 0.4.0 (250 rounds max)
cost 6 | cost 8 | cost 9 | cost 10 | cost 11 | cost 12 | cost 14 | cost 15 | |
---|---|---|---|---|---|---|---|---|
FavreBcrypt | 36.47 ms | 141.48 ms | 282.14 ms | 562.0 ms | 1122.04 ms | 2242.84 ms | 8985.64 ms | 17942.12 ms |
JBcrypt | 36.08 ms | 142.58 ms | 284.33 ms | 567.27 ms | 1133.36 ms | 2265.74 ms | 9096.0 ms | 18125.28 ms |
BC | 56.6 ms | 222.89 ms | 444.54 ms | 886.62 ms | 1773.0 ms | 3546.03 ms | 14178.62 ms | 28354.52 ms |
- Hardware: Moto G
- Software: SDK 22, LPBS23.13-17.3-1, Linux (3.4.42-g1bf0c10-00008-g04c189a)
- Tool-Version: 0.4.0 (250 rounds max)
cost 6 | cost 8 | cost 9 | cost 10 | cost 11 | cost 12 | cost 14 | cost 15 | |
---|---|---|---|---|---|---|---|---|
FavreBcrypt | 50.5 ms | 180.92 ms | 356.87 ms | 707.67 ms | 1410.14 ms | 2807.7 ms | 11228.48 ms | 22422.16 ms |
JBcrypt | 56.4 ms | 216.3 ms | 428.84 ms | 855.26 ms | 1706.08 ms | 3417.44 ms | 13632.92 ms | 27270.0 ms |
BC | 76.16 ms | 295.14 ms | 586.58 ms | 1170.04 ms | 2340.07 ms | 4675.33 ms | 18694.74 ms | 37446.76 ms |
- Hardware: i7-7700K, 24GB Ram
- Software: Android Emulator (HAXM), SDK 27, Java 1.8.0_172 (Oracle Corporation), Windows 10 (10.0)
- Tool-Version: 0.4.0 (250 rounds max)
cost 6 | cost 8 | cost 9 | cost 10 | cost 11 | cost 12 | cost 14 | cost 15 | |
---|---|---|---|---|---|---|---|---|
FavreBcrypt | 6.56 ms | 15.16 ms | 30.41 ms | 58.9 ms | 120.32 ms | 234.25 ms | 941.22 ms | 1805.56 ms |
JBcrypt | 3.86 ms | 15.56 ms | 30.02 ms | 60.78 ms | 122.66 ms | 242.26 ms | 961.44 ms | 1830.48 ms |
BC | 4.64 ms | 17.95 ms | 36.48 ms | 72.56 ms | 146.56 ms | 286.69 ms | 1098.52 ms | 2149.92 ms |
The sub-module benchmark-jmh
contains a simple JMH micro benchmark suite.
Win 10
# JMH version: 1.21
# VM version: JDK 1.8.0_172, Java HotSpot(TM) 64-Bit Server VM, 25.172-b11
# Run complete. Total time: 00:10:31
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark (cost) Mode Cnt Score Error Units
BcryptBenchmark.benchmarkBcBcryptLog 5 avgt 3 1,592 ± 0,036 ms/op
BcryptBenchmark.benchmarkBcBcryptLog 8 avgt 3 12,488 ± 0,078 ms/op
BcryptBenchmark.benchmarkBcBcryptLog 10 avgt 3 49,857 ± 0,269 ms/op
BcryptBenchmark.benchmarkBcBcryptLog 12 avgt 3 200,578 ± 25,269 ms/op
BcryptBenchmark.benchmarkFavreBcryptLog 5 avgt 3 1,759 ± 0,208 ms/op
BcryptBenchmark.benchmarkFavreBcryptLog 8 avgt 3 13,599 ± 0,297 ms/op
BcryptBenchmark.benchmarkFavreBcryptLog 10 avgt 3 54,062 ± 0,635 ms/op
BcryptBenchmark.benchmarkFavreBcryptLog 12 avgt 3 215,526 ± 6,247 ms/op
BcryptBenchmark.benchmarkJBcryptLog 5 avgt 3 1,774 ± 0,143 ms/op
BcryptBenchmark.benchmarkJBcryptLog 8 avgt 3 13,222 ± 0,135 ms/op
BcryptBenchmark.benchmarkJBcryptLog 10 avgt 3 52,583 ± 1,449 ms/op
BcryptBenchmark.benchmarkJBcryptLog 12 avgt 3 209,182 ± 4,118 ms/op
# JMH version: 1.21
# VM version: JDK 1.8.0_172, Java HotSpot(TM) 64-Bit Server VM, 25.172-b11
# Warmup: 2 iterations, 5 s each
# Measurement: 3 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
Benchmark (cost) Mode Cnt Score Error Units
BcryptBenchmark.benchmarkBcBcrypt 5 avgt 3 2,376 ± 0,263 ms/op
BcryptBenchmark.benchmarkBcBcrypt 6 avgt 3 4,814 ± 3,855 ms/op
BcryptBenchmark.benchmarkBcBcrypt 8 avgt 3 19,985 ± 11,735 ms/op
BcryptBenchmark.benchmarkBcBcrypt 10 avgt 3 76,259 ± 13,979 ms/op
BcryptBenchmark.benchmarkBcBcrypt 12 avgt 3 305,592 ± 28,062 ms/op
BcryptBenchmark.benchmarkBcBcrypt 14 avgt 3 1220,479 ± 136,330 ms/op
BcryptBenchmark.benchmarkFavreBcrypt 5 avgt 3 2,635 ± 0,712 ms/op
BcryptBenchmark.benchmarkFavreBcrypt 6 avgt 3 5,219 ± 1,119 ms/op
BcryptBenchmark.benchmarkFavreBcrypt 8 avgt 3 20,340 ± 1,963 ms/op
BcryptBenchmark.benchmarkFavreBcrypt 10 avgt 3 81,841 ± 1,818 ms/op
BcryptBenchmark.benchmarkFavreBcrypt 12 avgt 3 322,325 ± 35,615 ms/op
BcryptBenchmark.benchmarkFavreBcrypt 14 avgt 3 1354,536 ± 204,433 ms/op
BcryptBenchmark.benchmarkJBcrypt 5 avgt 3 2,746 ± 0,738 ms/op
BcryptBenchmark.benchmarkJBcrypt 6 avgt 3 5,321 ± 2,289 ms/op
BcryptBenchmark.benchmarkJBcrypt 8 avgt 3 20,993 ± 8,999 ms/op
BcryptBenchmark.benchmarkJBcrypt 10 avgt 3 87,351 ± 64,060 ms/op
BcryptBenchmark.benchmarkJBcrypt 12 avgt 3 335,343 ± 160,473 ms/op
BcryptBenchmark.benchmarkJBcrypt 14 avgt 3 1319,662 ± 587,548 ms/op
# JMH version: 1.21
# VM version: JDK 1.8.0_172, Java HotSpot(TM) 64-Bit Server VM, 25.172-b11
Benchmark (cost) Mode Cnt Score Error Units
BcryptBenchmark.benchmarkFavreBcrypt 5 avgt 3 2,736 ± 0,424 ms/op
BcryptBenchmark.benchmarkFavreBcrypt 6 avgt 3 5,378 ± 0,450 ms/op
BcryptBenchmark.benchmarkFavreBcrypt 8 avgt 3 21,416 ± 5,749 ms/op
BcryptBenchmark.benchmarkFavreBcrypt 10 avgt 3 84,175 ± 4,537 ms/op
BcryptBenchmark.benchmarkFavreBcrypt 12 avgt 3 335,873 ± 16,492 ms/op
BcryptBenchmark.benchmarkFavreBcrypt 14 avgt 3 1345,438 ± 40,216 ms/op
BcryptBenchmark.benchmarkJBcrypt 5 avgt 3 2,781 ± 0,267 ms/op
BcryptBenchmark.benchmarkJBcrypt 6 avgt 3 5,455 ± 0,077 ms/op
BcryptBenchmark.benchmarkJBcrypt 8 avgt 3 21,533 ± 0,833 ms/op
BcryptBenchmark.benchmarkJBcrypt 10 avgt 3 85,868 ± 8,942 ms/op
BcryptBenchmark.benchmarkJBcrypt 12 avgt 3 407,800 ± 2012,131 ms/op
BcryptBenchmark.benchmarkJBcrypt 14 avgt 3 1363,622 ± 11,306 ms/op