Refine vertical punctuation logic based on Unicode standard #3608
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Base the Unicode character blocks off of the official Unicode 9.0 character database. Refined the logic distinguishing upright, rotated, and neutral characters based on Unicode Technical Report 50 (with some simplifications). In particular, not everything in the General Punctuation block is treated as having neutral orientation; instead, the vertical punctuation table is consulted.
1ec5
force-pushed
the
1ec5-vertical-punctuation-tr
branch
from
d62da3e
to
0dc9bad
Compare
lucaswoj
approved these changes
Nov 14, 2016
| @@ -64,6 +65,7 @@ module.exports = { | |||
| // 'Batak': (char) => char >= 0x1BC0 && char <= 0x1BFF, | |||
| // 'Lepcha': (char) => char >= 0x1C00 && char <= 0x1C4F, | |||
| // 'Ol Chiki': (char) => char >= 0x1C50 && char <= 0x1C7F, | |||
| // 'Cyrillic Extended-C': (char) => char >= 0x1C80 && char <= 0x1C8F, | |||
There was a problem hiding this comment.
Any idea how I might've missed this and other blocks the first time around?
There was a problem hiding this comment.
This block is new to Unicode 9.0, whereas the file you consulted was based on Unicode 8.0. I've updated this file to point to the Unicode Character Database, which will make it easier for us to keep this file up-to-date in the future (by diffing between Unicode releases).
| //if (isChar['CJK Unified Ideographs Extension C'](char)) return true; | ||
| //if (isChar['CJK Unified Ideographs Extension D'](char)) return true; | ||
| //if (isChar['CJK Unified Ideographs Extension E'](char)) return true; | ||
| //if (isChar['CJK Compatibility Ideographs Supplement'](char)) return true; |
There was a problem hiding this comment.
I'd prefer if this code block were copied into mapbox/DEPRECATED-mapbox-gl#29 than left commented out in the codebase.
There was a problem hiding this comment.
See mapbox/DEPRECATED-mapbox-gl#29 (comment) for an updated list.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The list of Unicode character blocks is now based on the official Unicode 9.0 character database, whereas the previous table was based on an older Unicode standard.
The logic distinguishing upright, rotated, and neutral characters has been refined based on Unicode Technical Report #50 (proposed revision 16, which corresponds with Unicode 9.0).
Here are some example effects of this PR:
To keep things simple, the Letterlike Symbols and Number Forms blocks as a whole are considered neutrally oriented. We can refine that portion further if this simplification turns out to be problematic.
Depends on mapbox/mapbox-gl-test-suite#169 (although technically no test suite change was required because the differences were so minor).
/cc @lucaswoj @nickidlugash