Text shaping and font fallback

[hecrj]

wgpu_glyph uses glyph-brush, which in turn uses rusttype. While the current implementation is able to layout text quite nicely, it does not perform any text shaping.

Text shaping with font fallback is a necessary feature for any serious GUI toolkit. It unlocks support to truly localize your application, supporting many different scripts.

The only available library that does a great job at shaping is HarfBuzz, which is implemented in C++. skribo seems to be a nice HarfBuzz wrapper for Rust.

This feature will probably imply rewriting wgpu_glyph entirely, as caching will be more complicated and the API will probably need to ask for more data.

[hecrj]

After the report in #48, I did a bit of research on script-aware font fallback in font-kit and skribo. It looks like it's an unsolved problem for the time being. Here are some interesting links that may be useful in the future:

• Script-aware fallback servo/font-kit#37
• Script matching in skribo
• Interaction with CSS font matching algorithm linebender/skribo#22

[piaoger]

After the report in #48, I did a bit of research on script-aware font fallback in font-kit and skribo. It looks like it's an unsolved problem for the time being. Here are some interesting links that may be useful in the future:

* [servo/font-kit#37](https://github.com/servo/font-kit/issues/37)

* [Script matching in `skribo`](https://github.com/linebender/skribo/blob/master/docs/script_matching.md)

* [linebender/skribo#22](https://github.com/linebender/skribo/issues/22)

Recently, it seems that a new player is comming.

allsorts@github

blog

[dabreegster]

Something else to follow: a native Rust harfbuzz port, which will unlock vector text in WASM. Both by @RazrFalcon.

[RazrFalcon]

Hi! I hope to finish rustybuzz in 2-3 month, which should help a lot. But it doesn't include font fallback. I also need a proper font fallback in resvg, by I have no idea how it should be implemented. Afaik, the are no rules for this. Everyone is doing whatever they want. But it would be great to share some code instead of using an internal implementation.

I also plan to write a font matching library, similar to what fontkit does, but without system libraries.

[hecrj]

Hey @RazrFalcon, really looking forward to rustybuzz.

Afaik, the are no rules for this. Everyone is doing whatever they want. But it would be great to share some code instead of using an internal implementation.

My knowledge when it comes to this is very limited. I shared a couple of links in a previous comment, but I imagine you are probably aware of them already.

I agree having a shared solution would be great. Is the challenge here related to creating a solution that is decoupled from the font parsing/matching library?

I also plan to write a font matching library, similar to what fontkit does, but without system libraries.

Sounds great! Iced would benefit from this too. As of now, font-kit is pulling a bunch of external dependencies—like harfbuzz—even though we do not really shape any text.

Let me know if you have any ideas about how we could help you in your efforts.

[RazrFalcon]

I guess the best one I've saw is: https://raphlinus.github.io/rust/skribo/text/2019/04/04/font-fallback.html But it has more questions than answers.

Also, sort of related: https://lord.io/blog/2019/text-editing-hates-you-too/

Is the challenge here related to creating a solution that is decoupled from the font parsing/matching library?

The biggest problem is the algorithm itself. I'm not an expert either, but afaiu the algorithm involves:

1. Shaping with a master font.
2. Checking what glyphs are missing.
3. Finding a font(s) with those glyphs.
4. Shaping again.
5. Now, we have to somehow merge two(or more) shaping results.
   resvg simply replaces glyphs, which is totally incorrect.
   I guess it should be done on a clusters level or per-word.

Also, resvg shapes a whole text chunk (in terms of SVG), but maybe a per-word shaping will be better.

Right now I'm focused only on shaping and font matching, so a font fallback can be implemented on top of them.

Let me know if you have any ideas about how we could help you in your efforts.

Free testers are the best resource 😄

[SimonSapin]

As far as I understand there are two main approaches to font fallback:

• Shaper-driven fallback tha @RazrFalcon described above
• Using the cmap of fonts to find one that supports all code points in a given grapheme cluster, like in https://drafts.csswg.org/css-fonts/#font-matching-algorithm

Skribo aims to eventually provide one of those (not just wrap Harfbuzz) but as you found it’s not there yet.

[RazrFalcon]

@SimonSapin I'm not sure it's possible (or at least robust). HarfBuzz uses font-based, script-based text normalization, which checks not only cmap, but other tables too (mainly GDEF and GSUB).

[cryptoquick]

I noticed, this is on Druid's todo list as well.
linebender/druid#883

@RazrFalcon How "ready" is rustybuzz these days?

[MTRNord]

@dhardy

@ zumoshi kas-text is designed for integration with KAS. It could absolutely be used with Iced, but some parts of the API may not be ideal due to differences between the GUIs (stateful widgets vs not).

Hi I currently am writing a chat client so font fallbacks are a must for me to do emoji. Can you give any hint on how to use kas-text with iced or is that also a non simple job? (aka needs changes to iced).

[dhardy]

@MTRNord that would be quite a significant change to Iced. Also, KAS-text doesn't support emojis or embedded images yet.

[lj94093]

This issue is opened in 2019 and still being open.
Are there some people still concerening this issue?
I found a solution for font fallback in neovide if it could be helpful.

[unreal79]

Funny thing is: Iced version 0.3 supported Unicode characters. At least I'm able to set button's text to Cyrillic.

And version 0.4 used to support Unicode. I found this commit (https://github.com/iced-rs/iced/tree/15a13a76b4b0534d08afc0328b90267048e41b9d) is allowing me to use Cyrillic and Arabic characters (but not Chinese)... What happened? Why does Iced still use Strings (that support UTF-8 out of the box), but not Unicode?

[unreal79]

Ok, I investigated further. Looks like last commit that supports Unicode (Cyrillic) is https://github.com/iced-rs/iced/tree/1e3feee3a36f25d7e2eda231c3e6b895858952c5

Next commit https://github.com/iced-rs/iced/tree/825c7749ff364cf1f7ae5cab0c25f27ed85c7d82 breaks Unicode support.

[unreal79]

And... simply adding features = ["default_system_font"] to my Cargo.toml file fixed Unicode support for me...

[genusistimelord]

Funny thing is: Iced version 0.3 supported Unicode characters. At least I'm able to set button's text to Cyrillic.

And version 0.4 used to support Unicode. I found this commit (https://github.com/iced-rs/iced/tree/15a13a76b4b0534d08afc0328b90267048e41b9d) is allowing me to use Cyrillic and Arabic characters (but not Chinese)... What happened? Why does Iced still use Strings (that support UTF-8 out of the box), but not Unicode?

UTF-8 is a compressed version of Unicode. This is where the Unicode Char that is a 32bit variable is then transformed into a [u8] pattern based on the symbol used. For example, English symbols would fit into a single u8 while Japanese might only fit into [u8;3]. This is done to reduce the storage space needed to store certain Symbols which are more heavily used than other symbols.

Also yes for some reason he disabled the default_system_font from being set on default.

[SimonSapin]

And... simply adding features = ["default_system_font"] to my Cargo.toml file fixed Unicode support for me...

It sounds like the problem if your case was not Unicode support, but lacking a font that contains the relevant glyphs. In other words this was tofu, not mojibake.

[unreal79]

Thanks for the information about tofu/mojibake. In my defense I may add that my OS (Win 10) is fully Cyrillic and I don't expect to see tofu or mojibake characters anywhere.

My concern is the disabled default_system_font. How can an Iced newbie know about non-standard font/glyphs been used by default?! It isn't mentioned in documentation. AFAIK adding the feature in Cargo.toml is the only way to actually use system font (setting default_font: None in application Settings didn't help, contrary to what docs say). I suspect many potential users abandoned Iced only because they couldn't use non-latin characters.

[hecrj]

@unreal79 PRs welcome 😉

[GyulyVGC]

Is there any alternative to display coloured emoji?
The only thing that comes to my mind is to have images and displaying them as icons but it's not feasible in my case (I need to display all the countries flags so it would not be a solution to have 200+ images around my code)

[hecrj]

Text shaping and font fallback are landing for Canvas in #1610.