Localization fails with .utf8 suffixed locales

[ColinPitrat]

On my system, by default, the locale is:

LANG=fr_FR.utf8

If I run a druid UI with it, it fails to localize using fr-FR locales:

$ echo $LANG
fr_FR.utf8
$ ./target/debug/rust-ui-druid
DEBUG [druid::localization] available locales [en-US, fr-FR], current en-US
DEBUG [druid::localization] resolved: [en-US]

It works fine with LANG=fr_FR:

$ LANG=fr_FR ./target/debug/rust-ui-druid
DEBUG [druid::localization] available locales [en-US, fr-FR], current fr-FR
DEBUG [druid::localization] resolved: [fr-FR, en-US]

It would be nice to:

• Have more logs so that understanding what happens is easier (e.g which value is really found before any defaulting)
• Ignore the .utf8 suffix

[ColinPitrat]

Looking at the code, resolving the variable happens here:
https://github.com/linebender/druid/blob/master/druid-shell/src/backend/x11/application.rs#L673-L682

        var_non_empty("LANGUAGE")
            // the LANGUAGE value is priority list seperated by :
            // See: https://www.gnu.org/software/gettext/manual/html_node/The-LANGUAGE-variable.html#The-LANGUAGE-variable
            .and_then(|locale| locale.split(':').next().map(String::from))
            .or_else(|| var_non_empty("LC_ALL"))
            .or_else(|| var_non_empty("LC_MESSAGES"))
            .or_else(|| var_non_empty("LANG"))
            .unwrap_or_else(|| "en-US".to_string())

But the main problem comes from parsing the variable which is done in:
https://github.com/linebender/druid/blob/master/druid/src/localization.rs#L211-L213

        let default_locale: LanguageIdentifier =
            "en-US".parse().expect("failed to parse default locale");
        let current_locale = Application::get_locale()
            .parse()
            .unwrap_or_else(|_| default_locale.clone());

So it seems the problem is actually in unic_langid crate.
This crate is to parse unicode language IDs (https://unicode.org/reports/tr35/tr35.html#Unicode_language_identifier) which is not what is used in locale variables.
Locale variables rather have the format (https://docs.oracle.com/cd/E23824_01/html/E26033/glmbx.html):

language[_territory][.codeset][@modifier]

It seems a more appropriate crate to parse the locale would be https://docs.rs/locale-types/0.4.0/locale_types/locale/index.html

[JAicewizard]

It seems to fail basically always. I have 3 locales enabled, en_US.utf8, en_GB.utf8 and nl_NL.utf8, and none of them listed in by druid.

[JAicewizard]

@ColinPitrat can you confirm this fixes it? Since there are multiple backends to take care of, I dont think we can just 123 switch to another library, that might have unintended consequences. There was already something to split the locale on macos, this extends that to X11, and GTK

[ColinPitrat]

I didn't test it yet but I expect this would fix my problem indeed. Added a few nit comments.