Exception when reading m4b files metadata via new Track(...)

[sandreas]

The problem

If a Track is loaded with 4.5.0, the following exception is thrown (with 4.3.0 everything works fine).

dependency

      <PackageReference Include="z440.atl.core" Version="4.5.0" />
<!-- works: <PackageReference Include="z440.atl.core" Version="4.3.0" /> -->

Code

var t = new Track("any-m4b-file");

Exception

The type initializer for 'Commons.Utils' threw an exception.
   at Commons.Utils.get_Latin1Encoding()
   at ATL.AudioData.IO.ID3v1.ReadTag(BufferedBinaryReader source)
   at ATL.AudioData.IO.ID3v1.read(BinaryReader source, ReadTagParams readTagParams)
   at ATL.AudioData.IO.MetaDataIO.Read(BinaryReader source, ReadTagParams readTagParams)
   at ATL.AudioData.AudioDataManager.read(BinaryReader source, ReadTagParams readTagParams)
   at ATL.AudioData.AudioDataManager.read(BinaryReader source, Boolean readEmbeddedPictures, Boolean readAllMetaFrames, Boolean prepareForWriting)
   at ATL.AudioData.AudioDataManager.ReadFromFile(Boolean readEmbeddedPictures, Boolean readAllMetaFrames)
The type initializer for 'Commons.Utils' threw an exception.
   at Commons.Utils.get_Latin1Encoding()
   at ATL.AudioData.IO.ID3v1.ReadTag(BufferedBinaryReader source)
   at ATL.AudioData.IO.ID3v1.read(BinaryReader source, ReadTagParams readTagParams)
   at ATL.AudioData.IO.MetaDataIO.Read(BinaryReader source, ReadTagParams readTagParams)
   at ATL.AudioData.AudioDataManager.read(BinaryReader source, ReadTagParams readTagParams)
   at ATL.AudioData.AudioDataManager.read(BinaryReader source, Boolean readEmbeddedPictures, Boolean readAllMetaFrames, Boolean prepareForWriting)
   at ATL.AudioData.AudioDataManager.ReadFromFile(Boolean readEmbeddedPictures, Boolean readAllMetaFrames)
The type initializer for 'Commons.Utils' threw an exception.
   at Commons.Utils.get_Latin1Encoding()
   at ATL.AudioData.IO.ID3v1.ReadTag(BufferedBinaryReader source)
   at ATL.AudioData.IO.ID3v1.read(BinaryReader source, ReadTagParams readTagParams)
   at ATL.AudioData.IO.MetaDataIO.Read(BinaryReader source, ReadTagParams readTagParams)
   at ATL.AudioData.AudioDataManager.read(BinaryReader source, ReadTagParams readTagParams)
   at ATL.AudioData.AudioDataManager.read(BinaryReader source, Boolean readEmbeddedPictures, Boolean readAllMetaFrames, Boolean prepareForWriting)
   at ATL.AudioData.AudioDataManager.ReadFromFile(Boolean readEmbeddedPictures, Boolean readAllMetaFrames)

Environment

• 4.5.0
• worked with: 4.3.0
• Environment: macOS 10.15.7 / Ubuntu Linux 18.04. LTS (both tested)

If you need an m4b-file, that produces this issue, I'll send you one... (in my usecase, it happend with every m4b)

[Zeugma440]

Could you please use debug mode and report the InnerException property of the TypeInitializationException you've just reoprted ? The stacktrace doesn't give enough details.

NB : I haven't been able repro that on my side so far

[sandreas]

NB : I haven't been able repro that on my side so far

Yeah that is very strange. Running a raw command or the source code does not produce this exception on the same system with the same code and the same file. It is only within the tone project, although I'm not sure, what this can be.

It makes tracing the problem VERY hard, because the exception is not thrown in a catchable way but only printed. I'm using the IDE JetBrains Rider on Linux (Ubuntu 18.04 LTS) - as an example this does not catch an exception:

try {
    var t = new Track("any-m4b-file");
} catch(Exception e) {
// do nothing
}

but still printing it. I'm on it, if I can provide more info, I'll update this issue. May take some time.

[Zeugma440]

The exception printing might be the one happening in AudioDataManager.

I can make it print the InnerException in the next release, if that helps.

[Zeugma440]

Today's v4.06 logs the InnerException whenever it exists. I hope this will help. Please keep me informed.

[sandreas]

Today's v4.06 logs the InnerException whenever it exists. I hope this will help. Please keep me informed.

Yeah this worked, thank you. If the project is configured to use invariant culture (https://aka.ms/GlobalizationInvariantMode), the problem shows up:

<Project Sdk="Microsoft.NET.Sdk">
    <PropertyGroup>
        <!-- ... -->
        <InvariantGlobalization>true</InvariantGlobalization>
    </PropertyGroup>
</Project>

Setting it to false also removes the error. That is why it only showed up in the tone project and you weren`t able to reproduce it.

I tend to activate this setting in my public apps, because it makes the binaries much smaller and I normally don't need globalization support. For more details, read https://github.com/dotnet/runtime/blob/main/docs/design/features/globalization-invariant-mode.md

Now you have to decide, if you would like to support that :-)

For the sake of completeness, here is the trace.

The type initializer for 'Commons.Utils' threw an exception.
   at Commons.Utils.get_Latin1Encoding()
   at ATL.AudioData.IO.ID3v1.ReadTag(BufferedBinaryReader source)
   at ATL.AudioData.IO.ID3v1.read(BinaryReader source, ReadTagParams readTagParams)
   at ATL.AudioData.IO.MetaDataIO.Read(BinaryReader source, ReadTagParams readTagParams)
   at ATL.AudioData.AudioDataManager.read(BinaryReader source, ReadTagParams readTagParams)
   at ATL.AudioData.AudioDataManager.read(BinaryReader source, Boolean readEmbeddedPictures, Boolean readAllMetaFrames, Boolean prepareForWriting)
   at ATL.AudioData.AudioDataManager.ReadFromFile(Boolean readEmbeddedPictures, Boolean readAllMetaFrames)
Inner Exception BEGIN
Only the invariant culture is supported in globalization-invariant mode. See https://aka.ms/GlobalizationInvariantMode for more information. (Parameter 'name')
en is an invalid culture identifier.
   at System.Globalization.CultureInfo..ctor(String name, Boolean useUserOverride)
   at System.Globalization.CultureInfo..ctor(String name)
   at System.Globalization.CultureInfo.CreateSpecificCulture(String name)
   at Commons.Utils..cctor()
Inner Exception END
The type initializer for 'Commons.Utils' threw an exception.
   at Commons.Utils.get_Latin1Encoding()
   at ATL.AudioData.IO.ID3v1.ReadTag(BufferedBinaryReader source)
   at ATL.AudioData.IO.ID3v1.read(BinaryReader source, ReadTagParams readTagParams)
   at ATL.AudioData.IO.MetaDataIO.Read(BinaryReader source, ReadTagParams readTagParams)
   at ATL.AudioData.AudioDataManager.read(BinaryReader source, ReadTagParams readTagParams)
   at ATL.AudioData.AudioDataManager.read(BinaryReader source, Boolean readEmbeddedPictures, Boolean readAllMetaFrames, Boolean prepareForWriting)
   at ATL.AudioData.AudioDataManager.ReadFromFile(Boolean readEmbeddedPictures, Boolean readAllMetaFrames)
Inner Exception BEGIN
Only the invariant culture is supported in globalization-invariant mode. See https://aka.ms/GlobalizationInvariantMode for more information. (Parameter 'name')
en is an invalid culture identifier.
   at System.Globalization.CultureInfo..ctor(String name, Boolean useUserOverride)
   at System.Globalization.CultureInfo..ctor(String name)
   at System.Globalization.CultureInfo.CreateSpecificCulture(String name)
   at Commons.Utils..cctor()
Inner Exception END
The type initializer for 'Commons.Utils' threw an exception.
   at Commons.Utils.get_Latin1Encoding()
   at ATL.AudioData.IO.ID3v1.ReadTag(BufferedBinaryReader source)
   at ATL.AudioData.IO.ID3v1.read(BinaryReader source, ReadTagParams readTagParams)
   at ATL.AudioData.IO.MetaDataIO.Read(BinaryReader source, ReadTagParams readTagParams)
   at ATL.AudioData.AudioDataManager.read(BinaryReader source, ReadTagParams readTagParams)
   at ATL.AudioData.AudioDataManager.read(BinaryReader source, Boolean readEmbeddedPictures, Boolean readAllMetaFrames, Boolean prepareForWriting)
   at ATL.AudioData.AudioDataManager.ReadFromFile(Boolean readEmbeddedPictures, Boolean readAllMetaFrames)
Inner Exception BEGIN
Only the invariant culture is supported in globalization-invariant mode. See https://aka.ms/GlobalizationInvariantMode for more information. (Parameter 'name')
en is an invalid culture identifier.
   at System.Globalization.CultureInfo..ctor(String name, Boolean useUserOverride)
   at System.Globalization.CultureInfo..ctor(String name)
   at System.Globalization.CultureInfo.CreateSpecificCulture(String name)
   at Commons.Utils..cctor()
Inner Exception END

[Zeugma440]

Oh, so that's what it was !

I wonder if you can circumvent it by manually registering the ISO-8859-1 encoding : https://stackoverflow.com/a/37870346/8374722

Good news is that we found the core problem and its workaround. You're free to close the issue if you want.

[sandreas]

I wonder if you can circumvent it by manually registering the ISO-8859-1 encoding : https://stackoverflow.com/a/37870346/8374722

Thanks, but this did not work for me. Same error.

Good news is that we found the core problem and its workaround.

Well, kind of. The problem is based on a setting, that should FIX an issue, instead of creating one. The question is, if you would like atldotnet to further support an InvariantGlobalization or not (or maybe if it is even POSSIBLE because of the metadata string encodings). If NOT, it will be much more difficult to handle it on devices like Raspberry PI often not supporting ICU by default.

In atldotnet something has changed from 4.0.3 to 4.0.4, that requires globalisation modules (probably this part:

atldotnet/ATL/Utils/Utils.cs

Line 15 in 0a9eb25

private static Encoding latin1Encoding = Encoding.GetEncoding("ISO-8859-1");

). I don't know, if there is a possibility to overcome that while still providing the required support for character encodings.

For better understanding:

• I tend to build self-contained binaries for linux-arm platforms (e.g. Raspberry PI)
• self-contained binaries should work without ANY dependencies, but in fact there are some OS dependencies, that have to exist - luckily these are provided by default in the OS in most cases
• On most Raspberry PI OS, the icu package is missing by default or not available
• To deploy working binaries to those platforms, one probably would need the setting <InvariantGlobalization>true</InvariantGlobalization> (otherwise there is a segfault or a strange error message, which is hardly fixable)

I understand if you don't wanna go down this road, but maybe there is an easy fix without breaking something...

References:

• https://docs.microsoft.com/en-us/dotnet/core/deploying/trimming/trim-self-contained
• Couldn't find a valid ICU package installed on the system? dotnet/core#2186

[Zeugma440]

I totally share the goal of having a library that works on most platforms, including those where the icu package isn't available.

By examining the 4.03 -> 4.04 diff, I think I've located the culprit

This culture has been introduced to make sure date formatting doesn't depend on the host's locale, for a few cases where specs use US date formatting (e.g. legacy Audible expecting short months names such as "Aug").

It turns out the invariant culture gives the very same formatting, which should make us both happy.

=> Next release will only refer to CultureInfo.InvariantCulture instead of en-US 😄 Let's keep that issue open until you can confirm that does the trick.

[sandreas]

It turns out the invariant culture gives the very same formatting, which should make us both happy. => Next release will only refer to CultureInfo.InvariantCulture instead of en-US 😄 Let's keep that issue open until you can confirm that does the trick.

You sir, deserve a cookie. Thank you very much.

[Zeugma440]

Published in today's v4.07