fix(crypto): move FNV hashes from TypeScript to Rust/Wasm and implement iteration functionality

[jeremyBanks]

Moves std/crypto's FNV hash implementations from TypeScript into the Rust/Wasm bundle.

Background

The current implementation of the FNV hashing functions in std/crypto is implemented in TypeScript, unlike the other (cryptographic) hashing functions which are implemented in Rust/Wasm. The intent of this choice was to avoid the overhead of calling into Wasm, and to avoid the need of depending on third-party code for an algorithm that's simple enough to implement ourselves and for which we don't need to make any cryptographic guarantees.

Performance

Avoiding Wasm does yield performance benefits in some cases. On my machine, FNV32 hashing 64 bytes of data takes an average of 2 microseconds with either the TypeScript or Wasm implementations, but the Wasm performance is higher-variance, at least while it's initializing and warming up (with a 99th percentile performance of 10 microseconds and worst-case of 2 milliseconds). So if you're hashing a lot of very small values with FNV32/FNV32A, the TypeScript implementation can come out ahead.

However, in most other cases the situation is different. The TypeScript FNV64/FNV64A implementation incurs a lot of extra overhead from how it performs 64-bit integer operations using JavaScript's 64-bit float number type. Hashing 64 bytes of data takes an average of 8 microseconds in TypeScript versus 2 microseconds in Wasm. If the data becomes much larger, the TypeScript FNV performance abruptly goes off a cliff:

(Numbers from running the updated benchmark in crypto/_benches/bench.ts on my noisy janky machine, rounded for simplicity.)

• 64 B in FNV32: 2 microseconds in Wasm versus 2 microseconds in TypeScript (roughly equal)
• 256 KiB in FNV32: 300 microseconds in Wasm versus 6 milliseconds in TypeScript (~20x slower)
• 4 MiB in FNV32: 4 milliseconds in Wasm versus 100 milliseconds in TypeScript (~25x slower)
• 64 MiB in FNV32: 75 milliseconds in Wasm versus 1.6 seconds in TypeScript (~20x slower)
• 512 MiB in FNV32: 550 milliseconds in Wasm versus 13 seconds in TypeScript (~20x slower)

• 64 B in FNV64: 2 microseconds in Wasm versus 8 microseconds in TypeScript (~4x slower)
• 256 KiB in FNV64: 300 microseconds in Wasm versus 25 milliseconds in TypeScript (~80x slower)
• 4 MiB in FNV64: 4 milliseconds in Wasm versus 375 milliseconds in TypeScript (~90x slower)
• 64 MiB in FNV64: 75 milliseconds in Wasm versus 5.9 seconds in TypeScript (~80x slower)
• 512 MiB in FNV64: 575 milliseconds in Wasm versus 45 seconds in TypeScript (~80x slower)

This means that for many cases, using the current FNV implementation can be much slower than using one of the Wasm cryptographic hash implementations, when the reason it exists is supposed to be to provide a faster alternative for non-cryptographic use cases. This is an issue.

Maybe it's possible to optimize the TypeScript implementation to close this gap, but I'm not sure how to do that, while I do know how to move the code into our Rust/Wasm bundle, so that's what this PR does.

Correctness

The TypeScript FNV implementation has the separate issue that it currently doesn't support iterable inputs, such as Web Streams, but that's something our digest functions are meant to support and the Wasm/Rust implementations do (this PR resolves #3811). It can be refactored to handle this case, but when I did so that resulted in some duplication of logic and additional complexity that goes away when we move it into Rust/Wasm instead.

There are several existing implementations of FNV hashes on crates.io, however none of the widely-used ones provide all four variants we need. Since it's only a handful of lines of logic, I think it's safer just to implement it ourselves rather than add a new dependency on a not-widely-used crate, so that's what I've done here.

This PR merges the tests for FNV hashes with our tests for other hash functions, and adds the six test inputs specified in IETF draft-eastlake-fnv-21 and the four from Go's standard library tests to validate our implementation.

This change updates the crypto/_benches/bench.ts script to compare Wasm, WebCrypto, node:crypto, and TypeScript implementations, where applicable.

Size Impact

This change increases the size of deno_std_wasm_crypto.generated.mjs from 194,324 B to 196,614 B (a 1% increase of 2,290 B), or from 77,230 B to 78,068 B after gzip --best (a 1% increase of 838 B). (Note that this PR follows #4510 which reduces those initial sizes by 18% and 13% respectively, so the next release will still have a significant net reduction in Wasm size.)

The total size of the deleted TypeScript files implementing FNV hashes was 6252 B on disk.

Tangential Minor Breaking Change

This PR also renames the exported wasmDigestAlgorithms to DIGEST_ALGORITHM_NAMES, to match Deno's style guide and reflect the fact that it includes all algorithm names supported by the digest function, not just a subset. The exported WasmDigestAlgorithm type is removed, as it's now redundant with the existing DigestAlgorithmName export. (Generally, the fact that some algorithms are implemented in Wasm should probably be considered an implementation detail, and not exposed in the public interface.) The FNVAlgorithms export is also removed, since they're no longer a special case. (Note by @kt3k: This part is reverted in this PR).

[jeremyBanks]

(marked as draft because it includes/depends on changes from #4510, but this is mostly finished)

[jeremyBanks]

Here's my latest full output of deno bench crypto --filter=FNV. I'm not sure why the Wasm implementation has very high variance for FNV32A, while for FNV64A it has low variance and the TypeScript implementation has very high variance instead. Maybe it's something related to JIT warm-up, but I'm just guessing. However, I think the TypeScript implementation's extremely poor performance with non-tiny data is an overriding concern regardless of what's happening there.

As Text

cpu: Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz
runtime: deno 1.41.3 (x86_64-unknown-linux-gnu)

file:///mnt/c/Users/_/denoland/deno_std/crypto/_benches/bench.ts
benchmark                                              time (avg)        iter/s             (min … max)       p75       p99      p995
------------------------------------------------------------------------------------------------------- -----------------------------

TypeScript (from v0.220.1) with 64 B in FNV32A          2.29 µs/iter     436,906.1     (2.01 µs … 3.47 µs) 2.44 µs 3.47 µs 3.47 µs
Rust/Wasm with 64 B in FNV32A                           3.01 µs/iter     332,446.8       (2.2 µs … 2.4 ms) 2.6 µs 15 µs 16.9 µs

summary
  Rust/Wasm with 64 B in FNV32A
   1.31x slower than TypeScript (from v0.220.1) with 64 B in FNV32A

group 64 B in FNV64A
TypeScript (from v0.220.1) with 64 B in FNV64A          6.71 µs/iter     149,009.1     (5.4 µs … 967.7 µs) 6.1 µs 21.7 µs 26.6 µs
Rust/Wasm with 64 B in FNV64A                           1.44 µs/iter     696,684.8     (1.38 µs … 1.69 µs) 1.43 µs 1.69 µs 1.69 µs

summary
  Rust/Wasm with 64 B in FNV64A
   4.68x faster than TypeScript (from v0.220.1) with 64 B in FNV64A

group 256 KiB in FNV32A
TypeScript (from v0.220.1) with 256 KiB in FNV32A       6.36 ms/iter         157.2    (5.54 ms … 10.42 ms) 6.58 ms 10.42 ms 10.42 ms
Rust/Wasm with 256 KiB in FNV32A                      295.27 µs/iter       3,386.7     (273.4 µs … 478 µs) 299.5 µs 413.8 µs 448.9 µs

summary
  Rust/Wasm with 256 KiB in FNV32A
   21.54x faster than TypeScript (from v0.220.1) with 256 KiB in FNV32A

group 256 KiB in FNV64A
TypeScript (from v0.220.1) with 256 KiB in FNV64A       22.2 ms/iter          45.1   (21.38 ms … 24.89 ms) 22.24 ms 24.89 ms 24.89 ms
Rust/Wasm with 256 KiB in FNV64A                      315.01 µs/iter       3,174.5   (272.9 µs … 591.8 µs) 324.7 µs 450.4 µs 472.5 µs

summary
  Rust/Wasm with 256 KiB in FNV64A
   70.46x faster than TypeScript (from v0.220.1) with 256 KiB in FNV64A

group 4 MiB in FNV32A
TypeScript (from v0.220.1) with 4 MiB in FNV32A        98.95 ms/iter          10.1  (95.66 ms … 104.81 ms) 100.12 ms 104.81 ms 104.81 ms
Rust/Wasm with 4 MiB in FNV32A                          5.15 ms/iter         194.2     (4.54 ms … 6.15 ms) 5.43 ms 6.14 ms 6.15 ms

summary
  Rust/Wasm with 4 MiB in FNV32A
   19.22x faster than TypeScript (from v0.220.1) with 4 MiB in FNV32A

group 4 MiB in FNV64A
TypeScript (from v0.220.1) with 4 MiB in FNV64A        376.1 ms/iter           2.7 (353.45 ms … 444.37 ms) 378.96 ms 444.37 ms 444.37 ms
Rust/Wasm with 4 MiB in FNV64A                          4.77 ms/iter         209.6      (4.56 ms … 5.6 ms) 4.8 ms 5.32 ms 5.6 ms

summary
  Rust/Wasm with 4 MiB in FNV64A
   78.83x faster than TypeScript (from v0.220.1) with 4 MiB in FNV64A

group 64 MiB in FNV32A
TypeScript (from v0.220.1) with 64 MiB in FNV32A         1.74 s/iter           0.6       (1.62 s … 1.92 s) 1.78 s 1.92 s 1.92 s
Rust/Wasm with 64 MiB in FNV32A                         80.7 ms/iter          12.4   (76.25 ms … 85.21 ms) 83.29 ms 85.21 ms 85.21 ms

summary
  Rust/Wasm with 64 MiB in FNV32A
   21.57x faster than TypeScript (from v0.220.1) with 64 MiB in FNV32A

group 64 MiB in FNV64A
TypeScript (from v0.220.1) with 64 MiB in FNV64A         6.02 s/iter           0.2       (5.88 s … 6.26 s) 6.07 s 6.26 s 6.26 s
Rust/Wasm with 64 MiB in FNV64A                        81.36 ms/iter          12.3   (76.25 ms … 88.74 ms) 82.67 ms 88.74 ms 88.74 ms

summary
  Rust/Wasm with 64 MiB in FNV64A
   73.98x faster than TypeScript (from v0.220.1) with 64 MiB in FNV64A

group 512 MiB in FNV32A
TypeScript (from v0.220.1) with 512 MiB in FNV32A       13.46 s/iter           0.1      (12.99 s … 14.4 s) 13.93 s 14.4 s 14.4 s
Rust/Wasm with 512 MiB in FNV32A                      618.55 ms/iter           1.6  (609.8 ms … 626.73 ms) 621.93 ms 626.73 ms 626.73 ms

summary
  Rust/Wasm with 512 MiB in FNV32A
   21.75x faster than TypeScript (from v0.220.1) with 512 MiB in FNV32A

group 512 MiB in FNV64A
TypeScript (from v0.220.1) with 512 MiB in FNV64A       46.07 s/iter           0.0       (45.3 s … 48.5 s) 46.76 s 48.5 s 48.5 s
Rust/Wasm with 512 MiB in FNV64A                      597.24 ms/iter           1.7 (585.13 ms … 610.93 ms) 609.49 ms 610.93 ms 610.93 ms

summary
  Rust/Wasm with 512 MiB in FNV64A
   77.14x faster than TypeScript (from v0.220.1) with 512 MiB in FNV64A

As Image

If anyone wants to try the benchmark on a machine with more stable performance than mine, please share the results!

[iuioiua]

I like the overall idea of this PR! Excellent work. However, it introduces some breaking changes without first deprecating. We also prefer that a PR does a single thing. Let's do deprecations in a separate PR. You can search "@deprecated" to see how this is done elsewhere in the codebase.

[kt3k]

LGTM

[iuioiua]

LGTM too! Again, great work 👍🏾