Overlapping display of the upper vowel and the tone character in Thai Language

[nthana]

[Godot v. 3.1 stable]
[Tested on: Godot on MacOS]

In Thai language, a consonant character may be followed by an upper vowel, a tone character, or both. These characters must be displayed above the consonant. When these characters appear simultaneously, the tone character's vertical position should be moved to above the upper vowel character, preventing the overlap.

Currently, Godot can display upper vowels and tone characters correctly. However, when both occurs, they are incorrectly displayed in run-time (overlapping occur), but correctly displayed on the label-text editor in design-time.

The test project: https://github.com/nthana/ThaiFont

In label-text editor (correctly displayed):

On runtime (incorrectly displayed):

Thanks.

[akien-mga]

The difference between the in-game display and the editor display is due to the font Trirong-Medium.ttf. If you use the same font as editor font, you'll see the problem appear in the Inspector too.

The font itself is fine though, here's how it looks like in LibreOffice:

So there's an issue in how we/freetype handles this font.

[nthana]

@akien-mga Thank you. I'm grateful for the response. I'm totally agree with you. The font is fine, because I use this font with other applications. I also tried with other fonts, and the problem still occur.

I couldn't help on C++ coding because my knowledge on C++ & the Godot source code is limited.
But I am very glad to help on how the algorithm should be. With my Thai language knowledge, I will try to describe the algorithm on my next post. The algorithm is not too complex.

p.s. I'm native Thai and I've currently strived to promote Godot Engine usage in Thailand.

[nthana]

The below picture shows all of Thai characters in Unicode.
(adapt from source: https://jrgraphix.net/r/Unicode/0E00-0E7F )

Upper Vowels : shown in Blue frames.
Tone characters : shown in Red frame.

The algorithm is:

• whenever an "upper vowel" is followed by a "tone character", The tone character should be vertically raised to prevent overlapping.
• Special case: "Any Tone Character" + "0E33". In this case, the tone character should also be vertically raised to avoid overlapping.

(note: the only upper vowel that may follows a tone is "0E33")

Thanks.

[starry-abyss]

The algorithm is:

whenever an "upper vowel" is followed by a "tone character", The tone character should be vertically raised to prevent overlapping.

Not a languages guy, but I guess if there's an algorithm involved, one might need this feature #21791

[nthana]

@starry-abyss Wow! From the screen shot of #21791, it displays Thai language perfectly.

So, may be, this issue is the sub-problem of that general implementation in #21791

[akien-mga]

Since it works with the default editor font though (NotoSansThaiUI_Regular.ttf), it seems like we can already support the features needed for Thai even without advanced font shaping needed for e.g. Arabic.

Here's how Label looks like in game with NotoSansThaiUI_Regular.ttf (+ NotoSansUI_Regular.ttf as fallback to have the latin glyphs):

The diacritics are not shown on the two middle lines, which is weird as they are actually visible in the Inspector.

[akien-mga]

With both fonts, I can get the diacritics to show on the middle lines by changing the extra_spacing/char property of the DynamicFontData.

E.g. here with 1:

And with -3:

So yeah, there's definitely something fishy in the way DynamicFont handles those.

[nthana]

@akien-mga Thank you so much.

I think the reason that sometimes the diacritics are not shown is because there is no meaning in Thai when diacritics appear without preceded consonants. So, many font rendering engines will not render a diacritic alone. This also happen in many softwares such as Microsoft Office, and it is totally acceptable because any diacritic will never be used alone, in real-life sentence.

For the reason why the NotoSansThaiUI_Regular.ttf is displayed correctly, I think it is because its diacritics' positions are designed to always be at top most level to avoid overlapping with upper vowels. However, currently, many other Thai fonts place diacritics and vowels at the same vertical positions (because of aesthetic reason). So, these troubled fonts would need additional placement logics like in #21791.

In conclusion, at this time, if Thai language display is needed, the NotoSansThaiUI_Regular.ttf font should be used. However, for other Thai fonts, we should wait for the complex glyph positioning in #21791 to be completely implemented.

Thanks again :)

[nthana]

Just for information. I have additionally tested some Thai fonts and found that:

Some Thai Fonts that correctly displayed by Godot 3.1:

• Arial Unicode.ttf
• Ayuthaya.ttf
• Krungthep.ttf
• NotoSansThaiUI-Regular.ttf
• Sathu.ttf
• Silom.ttf

So, the worked fonts are much more than I previously understand.

Some Thai Fonts that incorrectly displayed by Godot 3.1 (Overlapping Occurs):

• Kanit-Regular.ttf
• Pridi-Regular.ttf
• Prompt-Regular.ttf
• Trirong-Medium.ttf

[bruvzg]

Should be fixed by CTL implementation (#41100):