Perf: String.replicate from O(n) to O(log(n)), up to 12x speed improvement

[abelbraaksma]

As in the title, plus it now doesn't create an unnecessary object anymore if the count is 0.

This is a follow-up on this comment, detailing the perf gains: #9390 (comment)

EDIT: I've changed the code a bit more, so that there are no unnecessary copies of strings anymore, it now gives even better perf than before:

[abelbraaksma]

VS Crash, again (after severe sluggishness and then grey-hanging), all I was doing was removing a comment line in FSharp.Core...

[KevinRansom]

This is pretty clever.

[realvictorprm]

I agree, nice work!

[Tarmil]

Is it really O(log(n)) though? You're still copying N*L bytes either way. I wonder whether the gains are more due to cache locality because you're copying from the target array rather than from the source string.

Regardless, nice improvement!

[realvictorprm]

That was my thought too but I guess it still is an improvement because of cache locality. Besides the numbers speak for themself here 🤔

[abelbraaksma]

Is it really O(log(n)) though? You're still copying N*L bytes either way. I wonder whether the gains are more due to cache locality because you're copying from the target array rather than from the source string.

@Tarmil, @realvictorprm, you are correct to a certain extend, and I was considering adding that to the description (I probably should have).

The Array.Copy function itself is optimized for certain block-sizes, and uses specific assembly instructions depending on that size. What I changed was the algorithm to copy the the char, which was using Append and was O(n) on the count. It is now O(log(n)) on the count and this allows the more efficient branches of Array.Copy to kick in once the doubling reaches a certain threshold. As a result, the gain in speed is sometimes 2x, sometimes 8x and occasionally O(1) if the count is 1 (the input reference is just returned in that case now, but this speed-up is not visible in the charts because of the allocation of the input).

It is impossible to change the O(n) behavior of copying memory in general, because, let's face it, copying is ultimately done bit by bit (or block by block if you consider vectorization) and will indeed behave O(n), where n is the target array or string size.

A faster method than Array.Copy exists for cases of contiguous memory (which is the case here), and that's the cpblk IL instruction. It has recently been sped up by some 300%. However, this instruction requires unsafe, which isn't allowed.

Btw, here's a benchmark (then made for ImmutableArray, see dotnet/runtime#32373) showing Array.Copy vs manual looping for small arrays: https://github.com/Therzok/BenchPlayground/blob/baf5a91e43f805ca413953c9bc749948627ed9cd/BenchPlayground/Benchmarks/runtime/ArrayCopy.Int32.Small.md

[abelbraaksma]

Btw, I ran those timings on dotnet Core, not .NET 5. dotnet/coreclr#27634 basically rewrites the happy path and greatly improves copying for small array sizes (up to 2.3x, apparently). Ultimately, current and new method end up using memmove for any sizable array.