CJK local glyph generation overrides custom text-font

[honjo2]

Updated to mapbox-ios-sdk-4.10.0.
Then text-font has been disabled.

"text-font": [
  "my-japanese-fontstack"
],

In this way PingFang is applied.

#13988

[julianrex]

/cc @fabian-guerra

[fabian-guerra]

Hi, @honjo2. Thank you for using Mapbox. I am sorry to hear you are having some issues. Could you please provide us with a code snippet or test app we could use to track down the issue?

Also, starting 4.10.0 we introduced MGLAttributedExpression that let's you set the text, font and scale size to MGLSymbolStyleLayer.text could you please try to set the font array using"

MGLSymbolStyleLayer.text = [NSExpression mgl_expressionForAttributedExpressions:@[[NSExpression expressionForConstantValue:attributedExpression]]];

[honjo2]

My web site using Mapbox GL JS

https://mapion.github.io/mapion-vector/#15/35.62277/139.75021

Style URL

https://vt01.mapion.co.jp/gl-server/style/lab.json?access_token=mt-pk.eyJ1IjoibWFwaW9uIiwiYSI6InJ5ZXFiOWJyZ3J2MjV3ZDcyNXY3Z3dmYzYifQ.XoOA7fe-5nUi8ALSyCl277

Using mapbox-ios-sdk-4.9.0 (no problem)

Using mapbox-ios-sdk-4.10.0 (There is a problem)

Problem

Why, only KANJI will be PingFang.
Japanese HIRAGANA and KATAKANA is OK.
Alphabet and Number too.

[fabian-guerra]

Hi, @honjo2. Thank you for your reply.

In my previous message asked you to try a workaround using MGLAttributedExpression object, do you think you could give it a try?

Is it possible you could share the style url and the account name where is uploaded? If you are not comfortable sharing the info here, you can send an email to customer support here https://support.mapbox.com/hc/en-us/requests/new?ticket_form_id=360000291231 and reference this ticket.

[honjo2]

MGLSymbolStyleLayer.text = [NSExpression mgl_expressionForAttributedExpressions:@[[NSExpression expressionForConstantValue:attributedExpression]]];

in Swift, please

[fabian-guerra]

Hi, @honjo2. This link has an example on how to create an attributed expression in Swift: https://docs.mapbox.com/ios/api/maps/4.10.0/Classes/MGLAttributedExpression.html

If you meant how to use it with the text then:
After retrieving your symbol style layer from your style:

symbolStyleLayer.text = NSExpression.init(forAttributedExpressions: [NSExpression(forConstantValue: attributedExpression)])

Could you please share with Mapbox support your account name, and style url so we can track down the issue:
https://support.mapbox.com/hc/en-us/requests/new?ticket_form_id=360000291231

In the ticket please reference this link.

[1ec5]

#13988 switched to local font rendering for all Han codepoints, including kanji in Japanese, and defaulted to a Simplified Chinese font. When local font rendering is enabled, the textFontNames property is ignored, because that property is only compatible with server-side fonts uploaded to Mapbox Studio.

Previously, local font rendering would only be enabled when the MGLIdeographicFontFamilyName property was set in Info.plist. You can avoid the default Chinese font by setting that Info.plist key to the name of a Japanese font you expect to be installed on the system. Unfortunately, there isn’t a way to turn off ideographic font rendering on iOS anymore. We should special-case a value like - or null to turn off local font rendering.

#7862 tracks using local font rendering for all codepoints in all languages. mapbox/mapbox-gl-js#3999 and mapbox/mapbox-gl-js#6197 discuss the need to account for language, not just script, in text processing.

[friedbunny]

Thanks, @1ec5. @fabian-guerra and I also did some testing:

• Reverting [ios] Make PingFang as the default local font family #13988 “fixes” the issue.
• Specifying a default CJK font will override the font for any Chinese characters.
  • This was also true before [ios] Make PingFang as the default local font family #13988 — if you set the MGLIdeographicFontFamilyName plist key, it would then always use that font for any Chinese characters.
  • PingFang does not have non-Chinese characters, so hiragata/katakana/hangul will be rendered in a different font.
  • We didn't find a workaround — setting MGLIdeographicFontFamilyName to an empty string didn’t work, for example. Bundling your custom font with your app could work, but we did not try this.
• In iosapp: PingFang TC is not a valid font family name on iOS — it should be PingFang or, better, Helvetica.

A few points:

• I think our local glyph generation being limited to “ideographic” characters is a mistake and leads to a poor experience for Japanese and Korean (the JK in CJK). We need to better support our non-Chinese customers/users and they should not be required to flip a setting to get proper text rendering.
• Local glyph generation happening when a style specifies a custom font is incorrect behavior.
• If we’re going to enable local glyph generation by default for CJK, it should be the default for all character sets — Allow using system fonts or fonts bundled in the app #7862.
• This issue can be observed in our default styles, too:

It’s visible in most labels here, but specifically: 丸の内 at the upper left should be rendered in the same typeface, but the hiragana の is in the custom font and 丸/内 are in PingFang.

[1ec5]

I think our local glyph generation being limited to “ideographic” characters is a mistake and leads to a poor experience for Japanese and Korean (the JK in CJK). We need to better support our non-Chinese customers/users and they should not be required to flip a setting to get proper text rendering.

Font substitution of Han characters causes more than aesthetic issues: due to Han unification, a number of characters have distinct glyph forms depending on the language (and to a lesser extent the region):

For native speakers, variants can be unintelligible or be unacceptable in educated contexts. English speakers may understand a handwritten note saying "4P5 kg" as "495 kg", but writing the nine backwards (so it looks like a "P") can be jarring and would be considered incorrect in any school. Likewise, to users of one CJK language reading a document with "foreign" glyphs: variants of 骨 can appear as mirror images, 者 can be missing a stroke/have an extraneous stroke, and 令 may be unreadable or be confused with 今 depending on which variant of 令 (e.g. 令󠄂) is used.

For the purposes of modern map display, hard-coding a Chinese font like PingFang primarily affects Japanese. (Han characters no longer appear in modern place names in Korean or Vietnamese.) There may also be a handful of edge cases in Chinese where a national variant of a character isn’t used where it should be.

There are two ways to disambiguate the characters, and both of them would require significant effort on the server side:

• The most common way is to use the right language-specific font for a given language and country. Unfortunately, Mapbox Streets source tiles contain no explicit language information for the name field, and there’s no machine-readable way for GL to know that the name_ja and name_zh fields contain Japanese and Chinese text, respectively (ref Document vector_layers extension tilejson-spec#14).
• Ideographic variation sequences could be inserted into the text in the vector tiles to force one variation or another. Unfortunately, both the fontstack API and GL currently assume a one-to-one relationship between characters and codepoints and between codepoints and glyphs. Another problem is that ideographic variation selectors are located in the Variation Selectors Supplement code block, which is in the unsupported Supplementary Special-Purpose Plane: Add support for rendering characters from unicode supplementary planes mapbox-gl-js#4001. The one-to-one relationship is a major downside of the current mechanism for client-side CJK rendering and the SDF-based approach to Allow using system fonts or fonts bundled in the app #7862. But if we rely on the system for text layout as well as text rendering, then ideographic variations sequences could be the solution.

Note that, depending on the system locale, operating systems often default to Simplified Chinese fonts in the absence of any language data.

/cc @mapbox/language

[m-stephen]

From my understanding, our first step is to set a default font for all character sets. It will fix two issues at least:

1. All characters are rendered by same font.
2. Still enable local glyph generation by default for CJK.

And next step, as @1ec5 mentioned above, might let sever join this game to improve this.

[friedbunny]

• From ios-v5.2.0-alpha.3 onward, specifying MGLIdeographicFontFamilyName = NO in your plist will allow the use of remote fonts for CJK glyphs.
• Hiragana and katakana are also now locally generated by default, since [core] Include Hiragana and Katakana glyph ranges to LocalGlyphRasterizer #15009.