Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add noble to benchmark #7

Merged
merged 5 commits into from
Feb 28, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,49 @@ const plainText2 = asImpl.open(key, nonce, sealed, true, ad);
expect(plainText2).to.be.deep.equal(plainText);
```

## Performance

Benchmark results of this implementation (chainSafe), stablelib 1.0.1 and noble 0.2.0 on a Mac M1 as of Aug 2023
```
chacha20poly1305
✓ chainsafe seal 32 bytes 721249.9 ops/s 1.386482 us/op - 21318 runs 30.1 s
✓ stablelib seal 32 bytes 144494.9 ops/s 6.920660 us/op - 4265 runs 30.0 s
✓ noble seal 32 bytes 150163.8 ops/s 6.659396 us/op - 4434 runs 30.0 s
✓ chainsafe open 32 bytes 671504.2 ops/s 1.489194 us/op - 19851 runs 30.1 s
✓ stablelib open 32 bytes 147999.8 ops/s 6.756768 us/op - 4363 runs 30.1 s
✓ noble open 32 bytes 157093.5 ops/s 6.365635 us/op - 4636 runs 30.0 s
✓ chainsafe seal 64 bytes 494830.5 ops/s 2.020894 us/op - 14610 runs 30.0 s
✓ stablelib seal 64 bytes 136125.4 ops/s 7.346169 us/op - 4025 runs 30.1 s
✓ noble seal 64 bytes 150779.6 ops/s 6.632199 us/op - 4459 runs 30.1 s
✓ chainsafe open 64 bytes 635232.8 ops/s 1.574226 us/op - 18745 runs 30.0 s
✓ stablelib open 64 bytes 145797.0 ops/s 6.858851 us/op - 4308 runs 30.1 s
✓ noble open 64 bytes 148018.9 ops/s 6.755892 us/op - 4371 runs 30.0 s
✓ chainsafe seal 1.00 KB 99285.50 ops/s 10.07196 us/op - 2924 runs 30.0 s
✓ stablelib seal 1.00 KB 38798.05 ops/s 25.77449 us/op - 1149 runs 30.1 s
✓ noble seal 1.00 KB 82057.41 ops/s 12.18659 us/op - 2424 runs 30.0 s
✓ chainsafe open 1.00 KB 100943.3 ops/s 9.906554 us/op - 2982 runs 30.1 s
✓ stablelib open 1.00 KB 36347.17 ops/s 27.51246 us/op - 1073 runs 30.0 s
✓ noble open 1.00 KB 80123.76 ops/s 12.48069 us/op - 2368 runs 30.1 s
✓ chainsafe seal 4.00 KB 28374.74 ops/s 35.24261 us/op - 837 runs 30.1 s
✓ stablelib seal 4.00 KB 11710.30 ops/s 85.39493 us/op - 347 runs 30.2 s
✓ noble seal 4.00 KB 30496.07 ops/s 32.79111 us/op - 900 runs 30.0 s
✓ chainsafe open 4.00 KB 29200.03 ops/s 34.24654 us/op - 861 runs 30.1 s
✓ stablelib open 4.00 KB 10933.00 ops/s 91.46624 us/op - 323 runs 30.1 s
✓ noble open 4.00 KB 30155.13 ops/s 33.16186 us/op - 892 runs 30.1 s
✓ chainsafe seal 8.00 KB 14577.58 ops/s 68.59850 us/op - 431 runs 30.2 s
✓ stablelib seal 8.00 KB 5880.412 ops/s 170.0561 us/op - 174 runs 30.3 s
✓ noble seal 8.00 KB 16014.05 ops/s 62.44518 us/op - 473 runs 30.1 s
✓ chainsafe open 8.00 KB 14990.32 ops/s 66.70972 us/op - 443 runs 30.1 s
✓ stablelib open 8.00 KB 5563.228 ops/s 179.7517 us/op - 165 runs 30.2 s
✓ noble open 8.00 KB 15821.22 ops/s 63.20624 us/op - 467 runs 30.1 s
✓ chainsafe seal 64.00 KB 1784.257 ops/s 560.4575 us/op - 54 runs 30.8 s
✓ stablelib seal 64.00 KB 735.6552 ops/s 1.359332 ms/op - 23 runs 32.6 s
✓ noble seal 64.00 KB 2020.801 ops/s 494.8534 us/op - 60 runs 30.7 s
✓ chainsafe open 64.00 KB 1854.636 ops/s 539.1893 us/op - 56 runs 30.8 s
✓ stablelib open 64.00 KB 691.5201 ops/s 1.446090 ms/op - 21 runs 31.8 s
✓ noble open 64.00 KB 2085.731 ops/s 479.4481 us/op - 62 runs 30.9 s
```

### License

Apache 2.0
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@
"@stablelib/chacha": "^1.0.1",
"@stablelib/poly1305": "^1.0.1",
"@stablelib/chacha20poly1305": "^1.0.1",
"@noble/ciphers": "^0.5.1",
"assemblyscript": "0.9.2",
"@dapplion/benchmark": "^0.2.2",
"@types/chai": "^4.2.15",
Expand Down
4 changes: 2 additions & 2 deletions test/perf/chacha20.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ describe("chacha20 (streamXOR) assemblyscript vs javascript", function () {
const nonceInplaceCounterLength = 4;

itBench({
id: `javascript with input length ${inputLength}`,
id: `stablelib ${inputLength}`,
before: () => ({key: new Uint8Array(key), nonce: new Uint8Array(nonce), input: new Uint8Array(input)}),
beforeEach: (v) => v,
fn: ({key: k, nonce: n, input: i}) => {
Expand All @@ -24,7 +24,7 @@ describe("chacha20 (streamXOR) assemblyscript vs javascript", function () {
});

itBench({
id: `assemblyscript with input length ${inputLength}`,
id: `chainsafe ${inputLength}`,
before: () => ({key: new Uint8Array(key), nonce: new Uint8Array(nonce), input: new Uint8Array(input)}),
beforeEach: (v) => v,
fn: ({key: k, nonce: n, input: i}) => {
Expand Down
73 changes: 38 additions & 35 deletions test/perf/chacha20poly1305.test.ts
Original file line number Diff line number Diff line change
@@ -1,57 +1,60 @@
import {itBench, setBenchOpts} from "@dapplion/benchmark";
import crypto from "crypto";
import {ChaCha20Poly1305 as ChaCha20Poly1305Stablelib} from "@stablelib/chacha20poly1305";
import {chacha20poly1305 as noble} from '@noble/ciphers/chacha';
import {ChaCha20Poly1305} from "../../src/chacha20poly1305";
import {KEY_LENGTH, NONCE_LENGTH, TAG_LENGTH} from "../../common/const";
import {KEY_LENGTH, NONCE_LENGTH} from "../../common/const";
import {newInstance} from "../../src/wasm";
import {formatBytes} from "./util";

describe("chacha20poly1305", function () {
this.timeout(0);
setBenchOpts({
minMs: 30_000,
});
const ctx = newInstance();
const asImpl = new ChaCha20Poly1305(ctx);
const chainsafe = new ChaCha20Poly1305(ctx);

const key = new Uint8Array(crypto.randomBytes(KEY_LENGTH));
const jsImpl = new ChaCha20Poly1305Stablelib(key);
const stablelib = new ChaCha20Poly1305Stablelib(key);
const nonce = new Uint8Array(crypto.randomBytes(NONCE_LENGTH));
// const jsImpl = new ChaCha20Poly1305Stablelib(key);
const ad = new Uint8Array(crypto.randomBytes(32));
const runsFactor = 1000;

for (const dataLength of [512, 4096, 16384, 65_536, 524_288]) {
const plainText = new Uint8Array(crypto.randomBytes(dataLength));
const sealed = jsImpl.seal(nonce, plainText, ad);

itBench({
id: `as seal with data length ${dataLength}`,
fn: () => {
asImpl.seal(key, nonce, plainText, ad);
},
});

itBench({
id: `js seal with data length ${dataLength}`,
fn: () => {
jsImpl.seal(nonce, plainText, ad);
},
});
for (const dataLength of [32, 64, 1024, 4096, 8192, 65536]) {
const plainText = new Uint8Array(crypto.randomBytes(dataLength));
const sealed = stablelib.seal(nonce, plainText, ad);
const testCases: {impl: string; sealFn: () => void; openFn: () => void}[] = [
{impl: "chainsafe", sealFn: () => chainsafe.seal(key, nonce, plainText, ad), openFn: () => chainsafe.open(key, nonce, sealed, ad)},
{impl: "stablelib", sealFn: () => stablelib.seal(nonce, plainText, ad), openFn: () => stablelib.open(nonce, sealed, ad)},
{impl: "noble", sealFn: () => noble(key, nonce, ad).encrypt(plainText), openFn: () => noble(key, nonce, ad).decrypt(sealed)}
]

itBench({
id: `as open with data length ${dataLength}`,
beforeEach: () => new Uint8Array(sealed),
fn: (clonedSealed) => {
// overwriteSealed as true to avoid memory allocation
asImpl.open(key, nonce, clonedSealed, ad, clonedSealed.subarray(0, clonedSealed.length - TAG_LENGTH));
},
});
// seal
for (const {impl, sealFn} of testCases) {
itBench({
id: `${impl} seal ${formatBytes(dataLength)}`,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Format in reverse to it's easier to read,

Suggested change
id: `${impl} seal ${formatBytes(dataLength)}`,
id: `${formatBytes(dataLength)} seal ${impl}`,

fn: () => {
for (let i = 0; i < runsFactor; i++) {
sealFn();
}
},
runsFactor,
});
}

itBench({
id: `js open with data length ${dataLength}`,
beforeEach: () => new Uint8Array(sealed),
fn: (clonedSealed) => {
jsImpl.open(nonce, clonedSealed, ad, clonedSealed.subarray(0, clonedSealed.length - TAG_LENGTH));
},
});
// open
for (const {impl, openFn} of testCases) {
itBench({
id: `${impl} open ${formatBytes(dataLength)}`,
fn: () => {
for (let i = 0; i < runsFactor; i++) {
openFn();
}
},
runsFactor,
});
}
}
});
9 changes: 9 additions & 0 deletions test/perf/util.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
export function formatBytes(bytes: number): string {
if (bytes < 1024) {
return bytes + " bytes";
} else if (bytes < 1024 * 1024) {
return (bytes / 1024).toFixed(2) + " KB";
} else {
return (bytes / (1024 * 1024)).toFixed(2) + " MB";
}
}
5 changes: 5 additions & 0 deletions yarn.lock
Original file line number Diff line number Diff line change
Expand Up @@ -531,6 +531,11 @@
"@jridgewell/resolve-uri" "^3.1.0"
"@jridgewell/sourcemap-codec" "^1.4.14"

"@noble/ciphers@^0.5.1":
version "0.5.1"
resolved "https://registry.yarnpkg.com/@noble/ciphers/-/ciphers-0.5.1.tgz#292f388b69c9ed80d49dca1a5cbfd4ff06852111"
integrity sha512-aNE06lbe36ifvMbbWvmmF/8jx6EQPu2HVg70V95T+iGjOuYwPpAccwAQc2HlXO2D0aiQ3zavbMga4jjWnrpiPA==

"@nodelib/[email protected]":
version "2.1.5"
resolved "https://registry.yarnpkg.com/@nodelib/fs.scandir/-/fs.scandir-2.1.5.tgz#7619c2eb21b25483f6d167548b4cfd5a7488c3d5"
Expand Down
Loading