Performance issue in my web app due to i18n

[Karthikeyan-tv]

Hi @turquoiseowl ,
I'm using i18n for the last 2 years in my web app and now we're working on the performance improvement.
While profiling my application, we found that the i18n module takes more CPU time compared to other modules. So we've removed i18n from the web app and load tested the app. We got the below results,

25 users with 5 minutes duration
With i18n,
Avg Response time: 348.7 ms
Request pre sec: 44.7RPS
CPU usage: 95%

Without i18n,
Avg Response time: 161.5 ms
Request pre sec: 70.2RPS
CPU usage: 50%

We're using the below keys in the web.config:

<add key="i18n.DirectoriesToScan" value=".." />
 <!-- Rel to web.config file -->
 <add key="i18n.WhiteList" value="*.cs;*.cshtml;*.sitemap;*.js" />
 <add key="i18n.BlackList" value=".\SampleReports;.\Scripts\AngularJS;.\Scripts\Bootstrap;.\Scripts\EssentialJS;.\Scripts\Jcrop;.\Scripts\jQuery;" />
 <add key="i18n.NuggetBeginToken" value="[[[" />
 <add key="i18n.NuggetEndToken" value="]]]" />
 <add key="i18n.NuggetDelimiterToken" value="||||" />
 <add key="i18n.NuggetCommentToken" value="////" />
 <add key="i18n.NuggetParameterBeginToken" value="(((" />
 <add key="i18n.NuggetParameterEndToken" value=")))" />

Are these results are normal or do you have any suggestion to improve the performance?

[turquoiseowl]

Thanks for your question. Those profile results are the first I've seen TBH but don't look unreasonable to me given the nature of the processing i18n is doing. Having said that I'm a bit disappointed in them because an effort was made to make i18n as performant as possible (in .NET) e.g. to reduce the number of object allocations during response processing and string manipulations.

I suppose you could drill down into different responses and number of nuggets contained to get an idea of per-request overhead vs. per nugget overhead. That might indicate areas for investigation.

[Karthikeyan-tv]

Hi @turquoiseowl ,

To ensure whether we are having issue only on our MVC application or all the MVC application, we have tested the MVC default template application with and without i18n and found below.

Scenario	CPU usage
without i18n	20%
with i18n (one message to be translatable)	21%
with i18n (around 100 message to be to be translatable)	95%

If you wish, I will attach the sample project which is hosted on azure for load testing

[turquoiseowl]

Okay, that is pretty firm confirmation that the CPU usage is related to the nugget message lookup.

I suspect if you can profile the actual process that most of the work will be in the i18n.TextLocalizer.LookupText method, and nested within that the call to ConcurrentDictionary.TryGetValue. If that is the case I'm stuck for potential optimization ideas as I chose that dictionary on the basis that it was the fastest/most-efficient.

If some other method shows up in a profiler then there may be scope for optimization.

[Karthikeyan-tv]

Hi @turquoiseowl,

Yes, it seems LookupText and EarlyUrlLocalizer taking more CPU time. I've attached the profiling results for your reference.

profile_60ecc4_w3wp_2476.zip

[turquoiseowl]

I'm having trouble looking at those results: VS2017, select full time range, expand w3wp.exe node it says 'No user code was running..."
Any chance of a screen shot? I'm particularly interested in what's going on in EarlyUrlLocalizer.

[Karthikeyan-tv]

Here is the screenshot,
LookupText:

EarlyUrlLocalizer:

[turquoiseowl]

Okay, thanks for that, I've also managed to interpret the results - the trick was to enable the Filter | Show External Code option.

It looks like there is, in fact, a problem with i18n's translation caching shown in your results. Essentially, i18n wraps the source PO file in a System.Web.Caching.CacheDependency instance which is then included in the cache lookup and allows any modification to that file to also bust the cache.

There's more than one possible reason I can see for the cache not working:

Firstly, it could be a side-effect of using the default language and NOT having a corresponding PO file for it. Rule this out by ensuring there is a PO file for the default language.

Secondly, it could be that the there's an issue with System.Web.Caching.CacheDependency in your server configuration. I vaguely remember an issue related to this before.

Or some other bug.

I've tested on a local IIS Express server and the cache is working fine so difficult for me to investigate further. If you are able to debug your server, I suspect you will find the call to i18n.TextLocalizer.LoadMessagesIntoCache is failing to make the insertion into the cache for the langtag due to the GetCacheDependencyForSingleLanguage call failing. Question would be why?

[turquoiseowl]

If your tests results were involving primarily translations in the i18n default language for the app, perhaps repeat the tests but with a language other than the default.

[Karthikeyan-tv]

Hi @turquoiseowl ,

Yes, you're correct. This high CPU usage is due to using the default language without the specific translation and it works well with specific translation files.

Thanks for your support!

[turquoiseowl]

Okay, thanks for reporting back.

I've modified the cache dependency code to be more sensible which will remove the difference between default and non-default languages. That is now on top of master branch and on Nuget as v2.1.12-pre002 if you could give it a try.

One other thing: you mentioned a sample webapp. That is something missing from the i18n project so would it be possible for you to contribute that project as a Pull Request? Would be most appreciated.