Update character data to Unicode 16.0.0

[1ec5]

The subset of Unicode character data in this repository hasn’t been updated since Unicode 12.0.0 in March 2019 (mapbox/mapbox-gl-js#8158). This PR updates it to Unicode 16.0.0, which will be released in about a month’s time. (All code points are already frozen.)

Since GL JS doesn’t yet support characters outside the Basic Multilingual Plane (#2307), the practical impact of this PR is negligible. The only practical impact is that this fixes the issue where an Arabic Extended-B character in a text-field prevented the surrounding Arabic text from being laid out from right to left. None of the characters in this code block are supported by mapbox-gl-rtl-text, so any affected character would continue to appear as a blank space. However, keeping this file up to date will facilitate any future support for characters outside the BMP. 😊

Launch Checklist

• Confirm your changes do not include backports from Mapbox projects (unless with compliant license) - if you are not sure about this, please ask!
• Briefly describe the changes in this PR.
• Link to related issues.
• Include before/after visuals or gifs if this PR includes visual changes.
• Write tests for all new functionality.
• Document any changes to public APIs.
• Post benchmark scores.
• Add an entry to CHANGELOG.md under the ## main section.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 66.66667% with 1 line in your changes missing coverage. Please review.

Project coverage is 87.63%. Comparing base (c207ccb) to head (9bde2c5).

Files	Patch %	Lines
src/util/script_detection.ts	50.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4536      +/-   ##
==========================================
+ Coverage   87.61%   87.63%   +0.01%     
==========================================
  Files         246      246              
  Lines       33451    33454       +3     
  Branches     2220     2217       -3     
==========================================
+ Hits        29309    29317       +8     
+ Misses       3125     3117       -8     
- Partials     1017     1020       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[HarelM]

Is there a way to cover this change with tests?
Also a chengelog entry would be welcomed here.
Thanks!

[1ec5]

Is there a way to cover this change with tests?

Maybe, how specific should the tests be? I hesitated to add any tests of the code block entries because that would result in hard-coding the code blocks in a second place. But I guess we could verify that each entry forms a range (end is greater than start)?

By the way, I would’ve written a script to generate this file automatically a long time ago, but the tricky part is automatically uncommenting each entry that is used somewhere in the codebase. Since Unicode only updates roughly annually, I haven’t bothered so far.

Also a chengelog entry would be welcomed here.

Ah OK, the contributing guide said not to add a changelog entry for internal tech debt updates, but I can add one.

[HarelM]

It has a change for Arabic users if I understand correctly, this should be the relevant part in the changelog.
A test that covers this change in terms of public API, code flow, something, IDK, even render test with special character.
Unless this doesn't change the behavior at all but only updated a "dependency".
In that case no test or changelog are needed.

[1ec5]

I added a trivial test of the function that determines whether text requires complex text shaping, as well as a test that entries in the Unicode code block table follow the expected format.

[HarelM]

THANKS!

[HarelM]

Bummber, merged it before a proper check of the place in the changelog, can you open a PR to fix this, i.e. move the changelog entry up to be under "main" branch?

[1ec5]

Oops, my bad, so that’s why there was a merge conflict. 😅 #4540