scalar GetIndexOfFirstNonAsciiByte

[Nick-Nuon]

Edit/NB: The benchmarks refer to the lastest commit, I shall edit them as I go

Here are the benchmarks on my home PC:

|                                       Method |    N |       Mean |     Error |    StdDev |
|--------------------------------------------- |----- |-----------:|----------:|----------:|
|                           FastUnicodeIsAscii |  100 |   764.9 ns |  14.56 ns |  13.62 ns |
|                       StandardUnicodeIsAscii |  100 | 2,852.7 ns |  98.46 ns | 290.32 ns |
|                               RuntimeIsAscii |  100 |   436.0 ns |   6.34 ns |   5.29 ns |
|            Error_GetIndexOfFirstNonAsciiByte |  100 |   212.4 ns |   3.02 ns |   2.67 ns |
|    Error_Runtime_GetIndexOfFirstNonAsciiByte |  100 |   181.7 ns |   3.60 ns |   5.49 ns |
|         allAscii_GetIndexOfFirstNonAsciiByte |  100 |   529.8 ns |  10.07 ns |  12.37 ns |
| allAscii_Runtime_GetIndexOfFirstNonAsciiByte |  100 |   334.1 ns |   6.41 ns |   7.87 ns |
|                           FastUnicodeIsAscii |  200 |   812.4 ns |  14.51 ns |  22.58 ns |
|                       StandardUnicodeIsAscii |  200 | 5,325.0 ns | 105.60 ns | 121.61 ns |
|                               RuntimeIsAscii |  200 |   655.5 ns |   3.45 ns |   2.69 ns |
|            Error_GetIndexOfFirstNonAsciiByte |  200 |   211.0 ns |   2.65 ns |   2.35 ns |
|    Error_Runtime_GetIndexOfFirstNonAsciiByte |  200 |   182.8 ns |   2.61 ns |   2.31 ns |
|         allAscii_GetIndexOfFirstNonAsciiByte |  200 |   830.6 ns |   6.21 ns |   5.50 ns |
| allAscii_Runtime_GetIndexOfFirstNonAsciiByte |  200 |   448.2 ns |   3.61 ns |   3.01 ns |
|                           FastUnicodeIsAscii |  500 | 1,537.7 ns |  13.50 ns |  12.63 ns |
|                       StandardUnicodeIsAscii |  500 | 6,120.5 ns |  74.17 ns |  65.75 ns |
|                               RuntimeIsAscii |  500 |   591.4 ns |   6.56 ns |   5.81 ns |
|            Error_GetIndexOfFirstNonAsciiByte |  500 |   219.5 ns |   4.35 ns |   4.27 ns |
|    Error_Runtime_GetIndexOfFirstNonAsciiByte |  500 |   181.4 ns |   1.73 ns |   1.44 ns |
|         allAscii_GetIndexOfFirstNonAsciiByte |  500 | 1,630.8 ns |  31.63 ns |  31.06 ns |
| allAscii_Runtime_GetIndexOfFirstNonAsciiByte |  500 |   861.0 ns |  14.40 ns |  13.47 ns |
|                           FastUnicodeIsAscii | 1000 | 3,020.5 ns |  57.82 ns |  54.08 ns |
|                       StandardUnicodeIsAscii | 1000 | 6,397.1 ns | 122.60 ns | 120.41 ns |
|                               RuntimeIsAscii | 1000 |   627.9 ns |  12.12 ns |  14.89 ns |
|            Error_GetIndexOfFirstNonAsciiByte | 1000 |   252.1 ns |   1.92 ns |   1.70 ns |
|    Error_Runtime_GetIndexOfFirstNonAsciiByte | 1000 |   224.7 ns |   1.78 ns |   1.66 ns |
|         allAscii_GetIndexOfFirstNonAsciiByte | 1000 | 3,132.3 ns |  17.41 ns |  16.29 ns |
| allAscii_Runtime_GetIndexOfFirstNonAsciiByte | 1000 | 1,583.3 ns |   6.86 ns |   5.73 ns |
|                           FastUnicodeIsAscii | 2000 | 6,912.5 ns | 100.41 ns | 140.76 ns |
|                       StandardUnicodeIsAscii | 2000 | 6,227.2 ns |  87.45 ns |  81.80 ns |
|                               RuntimeIsAscii | 2000 |   621.1 ns |   1.91 ns |   1.60 ns |
|            Error_GetIndexOfFirstNonAsciiByte | 2000 |   253.4 ns |   1.29 ns |   1.14 ns |
|    Error_Runtime_GetIndexOfFirstNonAsciiByte | 2000 |   219.9 ns |   1.68 ns |   1.40 ns |
|         allAscii_GetIndexOfFirstNonAsciiByte | 2000 | 6,371.9 ns | 123.37 ns | 164.70 ns |
| allAscii_Runtime_GetIndexOfFirstNonAsciiByte | 2000 | 2,993.9 ns |  31.95 ns |  29.89 ns |

// * Warnings *
MultimodalDistribution
  Checker.StandardUnicodeIsAscii: Default -> It seems that the distribution can have several modes (mValue = 3.1)
  Checker.RuntimeIsAscii: Default         -> It seems that the distribution can have several modes (mValue = 3.14)

I will ping soon, I get a nagging feeling about the benchmarks. I've taken a look around and probably have a few qs.

It is missing the benchmark for the runtime's GetIndexOfFirstNonAsciiByte . Not 100% sure how to get it.

[lemire]

It is missing the benchmark for the runtime's GetIndexOfFirstNonAsciiByte . Not 100% sure how to get it.

Possibly, we can copy-and-paste their code:

dotnet/runtime#89924

Note that they have vectorized versions.

A research question is whether testing whether GetIndexOfFirstNonAsciiByte finds nothing is slower than isAscii. These are equivalent tasks, but it is possible that isAscii is faster.

[Nick-Nuon]

It is missing the benchmark for the runtime's GetIndexOfFirstNonAsciiByte . Not 100% sure how to get it.

Possibly, we can copy-and-paste their code:

dotnet/runtime#89924

Still working on it. Copying the raw file directly made JIT complain that there exists missing code for this or that (which in turn created more missing code references), and I am hunting them down one by one, which doesn't sound very elegant but I'm not sure if there are other solutions.

I looked around a bit for others, and it may be because I don't know C# enough, but it seems that other methods of accessing a GetIndexOfFirstNonAsciiByte incur a performance cost (e.g. reflection) thus giving the baseline an unfair handicap?

Note that they have vectorized versions.

A research question is whether testing whether GetIndexOfFirstNonAsciiByte finds nothing is slower than isAscii. These are equivalent tasks, but it is possible that isAscii is faster.

OK thanks for the heads up, just forgot to include that one case.

[lemire]

Ping me if you want me to investigate. I will be glad to. :-)

[Nick-Nuon]

A research question is whether testing whether GetIndexOfFirstNonAsciiByte finds nothing is slower than isAscii. These are equivalent tasks, but it is possible that isAscii is faster.

it looks like isAscii is faster?

[Nick-Nuon]

Merging as per our Zoom discussion,

I have another PR this afternoon that overlaps this one. Just waiting for the benchmarks to be done.

[lemire]

Fantastic.