Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Xamarin.Android.Build.Tasks] Add $(AndroidEnableAssemblyCompression) (…
…#4686) Currently, Xamarin.Android supports compression of managed assemblies within the `.apk` if the app is built with [`$(BundleAssemblies)`=True][0], with the compressed assembly data stored within `libmonodroid_bundle_app.so` using gzip compression and placed in an array inside the data section of the shared library. There are two problems with this approach: 1. `mkbundle` emits C code, which requires a C compiler which requires the full Android NDK, and thus requires Visual Studio Enterprise. 2. Reliance on Mono's `mkbundle` results in possible issues around [filename globbing][1] such that `Xamarin.AndroidX.AppCompat.Resources.dll` is improperly treated as a [satellite assembly][2]. Because of (2), we are planning on [removing support][3] for `$(BundleAssemblies)` in .NET 6 ([née .NET 5][4]), which resulted in [some pushback][5] because `.apk` size is very important for some customers, and the startup overheads we believed to be inherent to `$(BundleAssemblies)` turned out to be somewhat over-estimated. To resolve the above issues, add an assembly compression mechanism that doesn't rely on `mkbundle` and the NDK: separately compress the assemblies and store the compressed data within the `.apk`. Compression is performed using the [managed implementation][6] of the excellent [LZ4][7] algorithm. This gives us a decent compression ratio and a much faster (de)compression speed than gzip/zlib offer. Also, assemblies are stored directly in the APK in their usual directory, which allows us to [**mmap**(2)][8] them in the runtime directly from the `.apk`. The build process calculates the size required to store the decompressed assemblies and adds a data section to `libxamarin-app.so` which causes *Android* to allocate all the required memory when the DSO is loaded, thus removing the need of dynamic memory allocation and making the startup faster. Compression is supported only in `Release` builds and is enabled by default, but it can be turned off by setting the `$(AndroidEnableAssemblyCompression)` MSBuild property to `False`. Compression can be disabled for an individual assembly by setting the `%(AndroidSkipCompression)` MSBuild item metadata to True for the assembly in question, e.g. via: <AndroidCustomMetaDataForReferences Include="MyAssembly.dll"> <AndroidSkipCompression>true</AssemblySkipCompression> </AndroidCustomMetaDataForReferences> The compressed assemblies still use their original name, e.g. `Mono.Android.dll`, so that we don't have to perform any string matching on the runtime in order to detect whether the assembly we are asked to load is compressed or not. Instead, the compression code *prepends* a short header to each `.dll` file (in pseudo C code): struct CompressedAssemblyHeader { uint32_t magic; // 0x5A4C4158; 'XALZ', little-endian uint32_t descriptor_index; // Index into an internal assembly descriptor table uint32_t uncompressed_length; // Size of assembly, uncompressed }; The decompression code looks at the `mmap`ed data and checks whether the above header is present. If yes, the assembly is decompressed, otherwise it's loaded as-is. It is important to remember that the assemblies are compressed at build time using LZ4 block compression, which requires assembly data to be entirely loaded into memory before compression; we do this instead of using the LZ4 frame format to make decompression at runtime faster. The compression output also requires a separate buffer, thus memory consumption at *build* time will be roughly 1.5x the size of the largest assembly, which is reused across all assemblies. ~~ Application Size ~~ A Xamarin.Forms "Hello World" application `.apk` shrinks by 27% with this approach for a single ABI: | Before (bytes) | LZ4 (bytes) | Δ | |------------------:|--------------:|:---------:| | 23,305,194 | 16,813,034 | -27.85% | Size comparison between this commit and `.apk`s created with `$(BundleAssemblies)` =True depends on the number of enabled ABI targets in the application. For each ABI, `$(BundleAssemblies)`=True creates a separate shared library, so the amount of space consumed increases by the size of the bundle shared library. The new compression scheme shares the compressed assemblies among all the enabled ABIs, thus effectively creating smaller multi-ABI `.apk`s. In the tables below, `mkbundle` refers to the APK created with `$(BundleAssemblies)`=True, `lz4` refers to the `.apk` build with the new compression scheme: | ABIs | mkbundle (bytes) | LZ4 (bytes) | Δ | |--------------------------------------:|------------------:|--------------:|---------| | armeabi-v7a, arm64-v8a, x86, x86_64 | 27,130,240 | 16,813,034 | -38.03% | | arm64-v8a | 7,783,449 | 8,746,878 | +11.01% | The single API case is ~11% larger because gzip offers better compression, at the cost of higher runtime startup overhead. ~~ Startup Performance ~~ When launching the Xamarin.Forms "Hello World" application on a Pixel 3 XL, the use of LZ4-compressed assemblies has at worst a ~1.58% increase in the Activity Displayed time (64-bit app w/ assembly preload enabled), while slightly faster on 32-bit apps, but is *always* faster than the mkbundle startup time for all configurations: | | | | | LZ4 vs | LZ4 vs | | Description | None (ms) | mkbundle (ms) | LZ4 (ms) | None Δ | mkbundle Δ | |----------------------------------:|----------:|--------------:|----------:|:--------:|:----------:| | preload enabled; 32-bit build | 795.8 | 855.6 | 783.8 | -0.25% ✓ | -7.22% ✓ | | preload disabled; 32-bit build | 777.1 | 843.0 | 780.5 | +0.44% ✗ | -7.41% ✓ | | preload enabled; 64-bit build | 779.0 | 843.0 | 791.5 | +1.58% ✗ | -6.82% ✓ | | preload disabled; 64-bit build | 776.0 | 841.6 | 781.5 | +0.69% ✗ | -7.15% ✓ | [0]: https://docs.microsoft.com/en-us/xamarin/android/deploy-test/release-prep/?tabs=windows#bundle-assemblies-into-native-code [1]: dotnet/android-libraries#64 [2]: https://github.com/mono/mono/blob/9b4736d4c271e9d4e04cafa258ddd58961f1a39f/mcs/tools/mkbundle/mkbundle.cs#L1315-L1317 [3]: dotnet/android-libraries#64 (comment) [4]: https://devblogs.microsoft.com/dotnet/announcing-net-5-preview-4-and-our-journey-to-one-net/ [5]: dotnet/android-libraries#64 (comment) [6]: https://www.nuget.org/packages/K4os.Compression.LZ4/ [7]: https://github.com/lz4/lz4 [8]: https://linux.die.net/man/2/mmap
- Loading branch information
