POTools localizable string extractor

[patrikrazem]

Hello,

I'm opening this issue simply because I have no other way of contacting you - you may delete this in the future, if it becomes obsolete.

I've been looking at different solutions for using GetText .po files within .NET projects and your solutions seem to be the most solid. A combination of your Karambolo.PO library, the text extraction tool and custom string localizers contained in this repository allow for a full-circle implementation of i18n through .po files -- extract texts, generate .po files, read localized strings from .po files through .NET Core built-in facilities.

While testing your approach, I've stumbled on a few questions and I was wondering if you'd be interested in addressing them:

1. Would you be interested in releasing the POTools project (now contained in this repo) as a separate repository, which could be publicly published on NuGet and thus easily available to be included in different kinds of projects as a .NET CLI Tool?
2. When extracting strings to be localized from .cs and .cshtml, is there a good reason why you're matching the strings based on the member name (T["Some string"]) instead of matching them based on the type of the member (IStringLocalizer)? Would that impact performance, or was that simply an arbitrary decision to do it that way?
3. When scanning files from which strings are extracted, without any special handling (something like scan | grep | extract), /bin/ files are taken into account, which is usually not desirable. Would it make sense to have an option to exclude certain directories within the specified path, or maybe even ignore the obvious directories like bin and obj?

I am posing these questions because I'm interested in building a solution that I can use throughout my different projects. I'm willing to contribute and invest some time into this, in order to find a universal solution that I can use long-term.

[adams85]

Hi there!

I'm glad that my localization solution has caught your attention (even despite the lack of docs). The implementation in this repo is like the third iteration of the concept, so I might say it's pretty polished.

My answers:

1. I've already thought about extracting the POTools project but I came to the conclusion that in its current state it's not generic enough to be worth distributing separately from this project. It'd need some more work to make it properly customizable so that it can cover a wider range of use cases. (E.g. it should be able to pick up translation texts from SPA or Blazor clients or, at least, provide some extensibility for consumers so that they can achieve things like that.) This may happen eventually but I can't yet see when, so I'd rather not make promises.

   As for now I suggest you copy POTools to your project and use it as a local tool as shown in this solution. The core idea is that tool projects are configured to emit NuGet packages to a solution directory when running dotnet pack and the output is used as a source to dotnet tool restore. Just examine the restore-tools.cmd script and you'll get a grasp of the idea. This setup even has some benefits over a published NuGet package: you can have different versions of these local tools adjusted to the actual project and you can make changes to them without much hassle at any time. You just issue restore-tools.cmd -f and you're ready to use the updated tools.

2. This decision has two main reasons:

   • Matching strings based on member names requires syntactic analysis only. To identify types, you have to do semantic analysis, which is a further step. Roslyn, the compiler API used by the tool, is capable of that, so it's absolutely doable. I just found it didn't provide much benefit, so didn't bother. I'm aware of only one bigger shortcoming of syntactic analysis: you must use exactly the expected names, qualified names won't work. I decided I could live with that compromise.
   • Another aspect is that I may have assemblies (e.g. API contract assemblies) which I don't want to make dependent on Microsoft.Extensions.Localization.Abstractions (the assembly containing IStringLocalizer) yet want to enable translation text extraction in those assemblies too. The current solution enables me to define a fake localizer interface for the purpose of extraction in such situations.

3. You can pass an MSBuild project file (e.g. csproj) to the scan command using the -p option (see dotnet po scan -? ) and it will then only list source files included in the project.

I hope this will help you. Just let me know if you have further questions.

[patrikrazem]

Thank you for your prompt answers, they're very helpful!

Re 1) and 2):
I'm not sure I would want to go down the whole SPA rathole - that feels like a whole different can of worms. I do like the idea of the tool being extendable, thought - maybe by allowing for arbitrary ILocalizableTextExtractor implementations to be registered/injected and allowing anyone to write an extractor for their specific needs.

I'll think about this some more and try to come up with a proof of concept when I have some free time. Maybe we can go from there.

Re 3):
I've tried doing that, but in my case, the tool simply gets stuck. I can, however, provide it a directory and it works fine. So basically: dotnet run scan --project-path ../SomeApp/ works, but dotnet run scan --project-path ../SomeApp/SomeApp.csproj does not. I am on macOS, running thi .NET Core 3.1.102.

I'll look into why this is, but it's probably something trivial.

[adams85]

I'll think about this some more and try to come up with a proof of concept when I have some free time. Maybe we can go from there.

Ok, the idea may be worth some exploration. However, I'm still not convinced that even an extensible tool published on NuGet would be a significant improvement over the solution we currently have. The possibility of quick and easy modification for tailoring the tool to the particular project's needs is a nice feature. By enabling extensions we'd achieve something similar but in that case the process of the customization seems to me somewhat more rigid and awkward. Anyway, feel free to play with this but please be aware that I can't really afford to maintain another project anytime soon.

I've tried doing that, but in my case, the tool simply gets stuck. I can, however, provide it a directory and it works fine. So basically: dotnet run scan --project-path ../SomeApp/ works, but dotnet run scan --project-path ../SomeApp/SomeApp.csproj does not. I am on macOS, running thi .NET Core 3.1.102.

I'm on Windows and have never encountered such an issue there, so I suspect it has something to do with macOS. The tool uses Buildalyzer to analyze MSBuild project files. Unfortunately, I don't have a Mac, so I can't do tests but first I'd try to upgrade Buildalyzer to the latest version and, if that doesn't solve it, would do some debugging around here.