Cultureinfo for Norwegian is not working in blazor webassembly .net5

[vsfeedback]

This issue has been moved from a ticket on Developer Community.

---

[severity:It's more difficult to complete my work]
Create a Blazor WebAssembly targeting .net5 and add edit the csproj to include true

Edit the Index.razor file:

@using System.Globalization
@page "/"

Problems with Globalization

@foreach (var language in languageCodes)
{
var culture = CultureInfo.CreateSpecificCulture(language);
var today = DateTime.Now.ToString("dddd, dd MMM yyyy", CultureInfo.GetCultureInfo(language));

}

Culture info from	Display Name	ThreeLetterISOLanguageNamem	ThreeLetterWindowsLanguageName	ToDay with format "dddd, dd MMM yyyy"
@language	@culture. DisplayName	@culture. ThreeLetterISOLanguageName	@culture. ThreeLetterWindowsLanguageName	@today

@code{
List languageCodes = new List { "no", "nb", "nb-NO", "nn", "sv-se", "de", "fr", "en-US", "da-DK" };

}

For the values "no", "nb", "nb-NO" and "nn" it does not seem to load the correct CultureInfo, and the days of the week have English names and the name of the months are displayed as the letter "M" followed by the number of the month.

[GlobalizationProblem.zip] (https://aka.ms/dc/file?name=B088da46e4652449ab05cf37260b79151637495230874692982_GlobalizationProblem.zip&tid=088da46e4652449ab05cf37260b79151637495230874692982)

---

Original Comments

Feedback Bot on 2/21/2021, 11:28 PM:

We have directed your feedback to the appropriate engineering team for further evaluation. The team will review the feedback and notify you about the next steps.

---

Original Solutions

(no solutions)

[pranavkm]

Have you configured your app to include all localization data: https://docs.microsoft.com/en-us/aspnet/core/blazor/globalization-localization?view=aspnetcore-5.0#blazor-webassembly?

[ghost]

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment. If it is closed, feel free to comment when you are able to provide the additional information and we will re-investigate.

See our Issue Management Policies for more information.

[paaleirik]

Have you configured your app to include all localization data: https://docs.microsoft.com/en-us/aspnet/core/blazor/globalization-localization?view=aspnetcore-5.0#blazor-webassembly?

Yes, and it works for other cultures like “sv-se”, “de”, “fr”, “en-US” and "da-DK”. The configuration is included in the attachment: GlobalizationProblem.zip
From the csproj-file:
<PropertyGroup> <TargetFramework>net5.0</TargetFramework> <BlazorWebAssemblyLoadAllGlobalizationData>true</BlazorWebAssemblyLoadAllGlobalizationData> </PropertyGroup>"

[paaleirik]

Any chance this issue will be re-investigate, or do I have to register a new issue?

[ghost]

Tagging subscribers to this area: @tarekgh, @safern
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This issue has been moved from a ticket on Developer Community.

---

[severity:It's more difficult to complete my work]
Create a Blazor WebAssembly targeting .net5 and add edit the csproj to include true

Edit the Index.razor file:

@using System.Globalization
@page "/"

Problems with Globalization

@foreach (var language in languageCodes)
{
var culture = CultureInfo.CreateSpecificCulture(language);
var today = DateTime.Now.ToString("dddd, dd MMM yyyy", CultureInfo.GetCultureInfo(language));

}

Culture info from	Display Name	ThreeLetterISOLanguageNamem	ThreeLetterWindowsLanguageName	ToDay with format "dddd, dd MMM yyyy"
@language	@culture. DisplayName	@culture. ThreeLetterISOLanguageName	@culture. ThreeLetterWindowsLanguageName	@today

@code{
List languageCodes = new List { "no", "nb", "nb-NO", "nn", "sv-se", "de", "fr", "en-US", "da-DK" };

}

For the values "no", "nb", "nb-NO" and "nn" it does not seem to load the correct CultureInfo, and the days of the week have English names and the name of the months are displayed as the letter "M" followed by the number of the month.

[GlobalizationProblem.zip] (https://aka.ms/dc/file?name=B088da46e4652449ab05cf37260b79151637495230874692982_GlobalizationProblem.zip&tid=088da46e4652449ab05cf37260b79151637495230874692982)

---

Original Comments

Feedback Bot on 2/21/2021, 11:28 PM:

We have directed your feedback to the appropriate engineering team for further evaluation. The team will review the feedback and notify you about the next steps.

---

Original Solutions

(no solutions)

Author:	vsfeedback
Assignees:	-
Labels:	area-System.Globalization, untriaged
Milestone:	-

[mkArtakMSFT]

@lewing FYI

[ghost]

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This issue has been moved from a ticket on Developer Community.

---

[severity:It's more difficult to complete my work]
Create a Blazor WebAssembly targeting .net5 and add edit the csproj to include true

Edit the Index.razor file:

@using System.Globalization
@page "/"

Problems with Globalization

@foreach (var language in languageCodes)
{
var culture = CultureInfo.CreateSpecificCulture(language);
var today = DateTime.Now.ToString("dddd, dd MMM yyyy", CultureInfo.GetCultureInfo(language));

}

Culture info from	Display Name	ThreeLetterISOLanguageNamem	ThreeLetterWindowsLanguageName	ToDay with format "dddd, dd MMM yyyy"
@language	@culture. DisplayName	@culture. ThreeLetterISOLanguageName	@culture. ThreeLetterWindowsLanguageName	@today

@code{
List languageCodes = new List { "no", "nb", "nb-NO", "nn", "sv-se", "de", "fr", "en-US", "da-DK" };

}

For the values "no", "nb", "nb-NO" and "nn" it does not seem to load the correct CultureInfo, and the days of the week have English names and the name of the months are displayed as the letter "M" followed by the number of the month.

[GlobalizationProblem.zip] (https://aka.ms/dc/file?name=B088da46e4652449ab05cf37260b79151637495230874692982_GlobalizationProblem.zip&tid=088da46e4652449ab05cf37260b79151637495230874692982)

---

Original Comments

Feedback Bot on 2/21/2021, 11:28 PM:

We have directed your feedback to the appropriate engineering team for further evaluation. The team will review the feedback and notify you about the next steps.

---

Original Solutions

(no solutions)

Author:	vsfeedback
Assignees:	-
Labels:	arch-wasm, area-System.Globalization
Milestone:	-

[lewing]

Which version of .NET are you testing, is this still a problem in .NET 6 preview 6?

[tarekgh]

@lewing please help answering @solbster and @catfiadhaich

[Solbstr]

@lewing , when will this be fixed?

[AbstractionsAs]

I just reported what seems to be the same issue yesterday, didn't see this one before today.
It's a major problem for us - and I imagine for a lot of other companies in Norway.

[juliank]

@lewing could you please triage these issue, I think @maryamariyan unintentionally moved it to 7.0?

@tarekgh: Is it intentional that this won't be fixed before 7.0? If so that is unfortunate, since 7.0 isn't an LTS which in practice means that many might have to wait for 8.0 to get this fixed.

[tarekgh]

Is it intentional that this won't be fixed before 7.0? If so that is unfortunate, since 7.0 isn't an LTS which in practice means that many might have to wait for 8.0 to get this fixed.

@lewing could you please help with @juliank question?

[catfiadhaich]

While I appreciate that there's a lot of work going on for version 7, the lack of even a comment about this issue from Mr. Ewing is disappointing. I'm sure there are bigger and more interesting fixes in progress for the next major release, but I think it's clear that this is a problem for a number on non-English culture configs.

[lewing]

The culture info we is use is produced from the dotnet/icu repository (a github fork of the standard ICU repo) and loaded by the browser runtime. Because we have to load all the information across the network (and it is very large we split) we have chosen to remove some information from the culture info and made a best effort at trading features for size. No choice to remove data is perfect and some kinds of applications are impacted more than others. Browsers themselves do something similar when shipping icu data.

Display names are a good example of something that increase the data size dramatically and are rarely used in most web apps so they are not currently included in our icu data bundle. https://unicode-org.github.io/icu/userguide/icu_data/buildtool.html#file-slicing-coarse-grained-features has a breakdown of the size impact of various parts of the data, and is part of the documentation for the tool we use to shape the data in the dotnet/icu build.

The good news is that for the most part if you bundle your application with a full icu data bundle all the missing features and data will just work (there are a couple of exceptions that would be relatively easy to fix). The bad news there isn't currently a pre-built mechanism to do this. We started work in .NET 6 to offer much finer grained control over the data bundle but it stalled and hasn't yet been restarted. I'll reopen discussion around this work internally but clear information about what data your applications need and why would be quite helpful in prioritizing work here.

[catfiadhaich]

Thank you for updating us on this, Larry. I can only speak for myself, obviously. I'm not blocked by this issue, but I would be interested in knowing more about the workaround that you mentioned because I really would like to use Blazor. it might take me a wee while to wrap my head around it, but I can figure its out and it might let me move on with using Blazor for my application.

I think others on this thread are affected way more than me... :)

In my case, I was concentrating mostly on Celtic languages (Irish Gaelige, Scottish Gaelic in particular) and my need is mostly for the correct formatting of dates, etc.and the ability to switch my UI between languages. The application is intended for education and (kinda) social media. While I could move to using another method for my presentation layer, Blazor seemed like the ideal solution for me.

[juliank]

@lewing asked:

clear information about what data your applications need and why would be quite helpful in prioritizing work here.

In our case the most important thing will be correctly localized names (for all the Nordic countries) on months and days of the week, for use in formatted datetime strings.

[igotinfected]

@lewing Similar case here. We have an app that app that supports en-US, fr-FR, de-DE and lb-LU, but none of the lb-LU seem to be localized. So when trying to display localized month and day values as returned by CultureInfo.CurrentCulture.DateTimeFormat.GetMonthName((int)month), the English values are displayed.

[ilonatommy]

I've been having similar problems with the "gd-GB" (Scottish Gaelic) and "cy-GB" (Welsh) cultures when using Blazor (wasm in this case - I have not tried this under server).

For example:

I placed the following code in the Program.cs file, right at the start of Main, and set the BlazorWebAssemblyLoadAllGlobalizationData property to true in the project file:

            Console.WriteLine(CultureInfo.CurrentCulture.DisplayName); 
            CultureInfo gd = new CultureInfo("cy-GB");
            Console.WriteLine(gd.DisplayName);                                   
            Console.WriteLine(DateTime.Now.ToString("f", gd));

When running under dotnet 3.1 (lts), the (developer tools) console shows the output of the first, third, and fourth lines.

English (United States) Welsh (United Kingdom) Dydd Sul, 5 Medi 2021 07:33

When setting the culture info to "gd-GB", I get what I'd expect.

English (United States) Scottish Gaelic (United Kingdom) DiDòmhnaich, 5mh dhen t-Sultain 2021 07:38

However, when using dotnet 5 or the latest (as of Sept 5) dotnet 6 preview (using VS2022), the results are... disappointing:

en (US) gd (GB) Sunday, September 5, 2021 7:43 AM

Even the DisplayName, etc. are incorrect. The results are similar for cy-GB.

I don't see this problem with an ASP.NET Core webapp or a console app. It seems fine when using those.

Please Note - I also see the same thing happening on macOS using dotnet 5. I have not yet tested this under dotnet6 preview on macOS.

Thank you @catfiadhaich, your problem falls under this issue: #44739. The reasons for shortening the names is provided there and is still under discussion.

[vinhch]

I had tested and found out a breaking change of Cultureinfo for Norwegian between .NetFW and .NetCore/.Net.
With .NetFW, Cultureinfo("nb-NO"), it has .Parent is Cultureinfo("nb"), and Cultureinfo("nb").Parent is Cultureinfo("no")
But with .NetCore/.Net, Cultureinfo("nb") has no parent at all

Here is a simple test and the results on dotnetfiddle.net, you guys can try it with other .Net version
.NetFW: https://dotnetfiddle.net/Q4i6PM
.Net6: https://dotnetfiddle.net/DwD9cy

using System;
using System.Globalization;
					
public class Program
{
	public static void Main()
	{
		ShowLanguageInheritance(new CultureInfo("nb-NO"));
	}
	
	public static void ShowLanguageInheritance(CultureInfo language)
	{
		if (IsNullOrHasNoParent(language))
		{
			return;
		}
		Console.WriteLine(language.Name);
		ShowLanguageInheritance(language.Parent);
	}
	
	public static bool IsNullOrHasNoParent(CultureInfo language)
	{
		return language == null || language.Name == language.Parent.Name;
	}
}

[tarekgh]

@vinhch this was issue on CLDR (which ICU using that Unicode Standard data from). unicode-org/cldr#1031. It is matter of picking a new ICU version.

[ilonatommy]

There are 4 different issues actually.

1. "nb-NO" and "nb" data was not available even though the locale was in filters and should be present in ICU bundle.
   Using the new icudt files from:
   https://github.com/ilonatommy/icu/tree/dotnet/main/eng/prebuilts
   fixes this problem. The bundle is not in the version shipped publicly, so a temporary workaround could be to exchange the files manually in the project for these in the repo linked above.
   I am closing the current issue and delegating other sub-problems to separate issues.

2. Canonical locale are not listed in icu filters. Locale considered canonical are listed here: locid.cpp. These are: "cy-GB", "is-IS", "bs-BA". The issue will be under discussion here:
   [wasm][icu] Canonical locales are not in filters but users would like to use them #79816

3. Aliases are not resolved properly. "no" is an alias for "nb" and should return the same data, see
   [wasm][icu] Locale aliases are not resolved correcly #79815

4. Non-canonical locale are missing. Such locale as "lb-LU", "gd-GB", "nn" are not considered canonical and are less frequently used. We are trying to cut down on data size as much as possible, so adding them in the near future is rather unlikely.

[igotinfected]

There are 4 different issues actually.

1. "nb-NO" and "nb" data was not available even though the locale was in filters and should be present in ICU bundle.

Using the new icudt files from:

https://github.com/ilonatommy/icu/tree/dotnet/main/eng/prebuilts

fixes this problem. The bundle is not in the version shipped publicly, so a temporary workaround could be to exchange the files manually in the project for these in the repo linked above.

I am closing the current issue and delegating other sub-problems to separate issues.

2. Canonical locale are not listed in icu filters. Locale considered canonical are listed here: locid.cpp. These are: "cy-GB", "is-IS", "bs-BA". The issue will be under discussion here:

#79816

3. Aliases are not resolved properly. "no" is an alias for "nb" and should return the same data, see

#79815

4. Non-canonical locale are missing. Such locale as "lb-LU", "gd-GB", "nn" are not considered canonical and are less frequently used. We are trying to cut down on data size as much as possible, so adding them in the near future is rather unlikely.

Thanks for the reply! I understand the need to cut down in size. Is there a way for us to include lb_LU in our Blazor app, akin to the temporary solution for the nb_NO one?

[ilonatommy]

There is a way to build your own, custom ICU data bundle. I will send you a custom one with all the locales + lb_LU on the email you pinned to your GH account. That's a screenshot of how it works for me:

In case anyone wants to build it themselves, here are the steps:

1. clone https://github.com/dotnet/icu
2. Go to icu-filters/icudt_browser.json if you're targeting browser or icu-filters/icudt_mobile.json if you're targeting mobile and add your locale in "localeFilter/whitelist". You can also remove some that you are sure you won't be using if you want to reduce size.
3. for Browser run the script in root folder:

EMSDK_PATH=$PWD/artifacts/emsdk
rm -rf $EMSDK_PATH
git clone https://github.com/emscripten-core/emsdk.git $EMSDK_PATH
EMSCRIPTEN_VERSION="`cat ./.devcontainer/emscripten-version.txt 2>&1`"
cd $EMSDK_PATH && ./emsdk install $EMSCRIPTEN_VERSION
cd $EMSDK_PATH && ./emsdk activate $EMSCRIPTEN_VERSION
cd ../../
./build.sh /p:TargetOS=Browser /p:TargetArchitecture=wasm /p:IcuTracing=true

for all Mobiles (not only Android and not only x64, you can use this configuration, it's the same) run the script:

export ANDROID_NDK_ROOT=$PWD/artifacts/ndk/
rm -rf $ANDROID_NDK_ROOT
mkdir  $ANDROID_NDK_ROOT
wget https://dl.google.com/android/repository/android-ndk-r25b-linux.zip
unzip android-ndk-r25b-linux.zip -d $ANDROID_NDK_ROOT
mv $ANDROID_NDK_ROOT/*/* $ANDROID_NDK_ROOT
rm android-ndk-r25b-linux.zip
rmdir $ANDROID_NDK_ROOT/android-ndk-r25b
./build.sh /p:TargetOS=Android /p:TargetArchitecture=x64 /p:IcuTracing=true

4. Go to artifacts/bin/icu-browser-wasm or the equivalent for mobiles and copy icudt.dat.
5. You can exchange your app's icudt.dat for the one you created. Keep in mind that each build will replace the file with the default one, so consider an after-build task of replacing it, we do not have a custom-ICU feature build-in yet.

[igotinfected]

Success!

To get your custom icudt.dat files to work for Blazor (WASM in my case), you need to hook into the build process and replace the reference to icudt.dat with your own, so your file is the one that gets copied into the output folder, otherwise you will run into integrity check issues.

To do that, add this to your main .csproj:

<!-- AFAIK there is no need to hook into the publish process, that should work automatically but I have not tested it -->
<Target Name="_ReplaceICU" AfterTargets="ResolveRuntimePackAssets">
  <Message Text="Replacing 'icudt.dat' file..." Importance="High" />

  <ItemGroup>
      <!-- Generate an item to match the item we need to remove from "ReferenceCopyLocalPaths" -->
      <ToRemove Include="toRemove">
          <DestinationSubPath>icudt.dat</DestinationSubPath>
      </ToRemove>

      <!-- Use said item to match against the real item we want to remove -->
      <ReferenceCopyLocalPaths Remove="@(ToRemove)"
                               MatchOnMetadata="DestinationSubPath"/>

      <!-- Now add the custom "icudt.dat" file -->
      <ReferenceCopyLocalPaths Include="$(MSBuildThisFileDirectory)icudt.dat"
                               DestinationSubPath="icudt.dat" />
     </ItemGroup>
</Target>

There might be a simpler way to achieve the same result but this is the best I could come up with given my limited knowledge of the msbuild processes.

Here's a working sample: https://github.com/igotinfected/test-icudt-blazor-wasm

@ilonatommy thank you for the file, and @javiercn thank you for pointing me in the right direction! 😊