Investigate UTF8/UTF16 transcoding with AVX512

[stephentoub]

And whether there's anything that can be applied from https://lemire.me/blog/2023/09/13/transcoding-unicode-strings-at-crazy-speeds-with-avx-512.

[ghost]

Tagging subscribers to this area: @dotnet/area-system-text-encoding
See info in area-owners.md if you want to be subscribed.

Issue Details

---

And whether there's anything that can be applied from https://lemire.me/blog/2023/09/13/transcoding-unicode-strings-at-crazy-speeds-with-avx-512.

Author:	stephentoub
Assignees:	-
Labels:	area-System.Text.Encoding, tenet-performance
Milestone:	9.0.0

[lemire]

I actually have a secret projet to work on this (not specifically with AVX-512, though that would be great).

We think that UTF-8 validation, for example, could be improved in dotnet.

If you'd like to chat, get in touch!!!

(Note that I have contributed to dotnet/runtime in the past.)