Skip to content

Commit

Permalink
[WIP] new Profiled AOT support
Browse files Browse the repository at this point in the history
Context: dotnet/runtime#71787
Context: https://dev.azure.com/dnceng/public/_artifacts/feed/7.0.100-preview.7.22377.5-nonshipping/NuGet/dotnet-pgo/overview/7.0.0-preview.7.22375.6

1. Record a `.nettrace` file:

    adb reverse tcp:9000 tcp:9001
    adb shell setprop debug.mono.profile '127.0.0.1:9000,suspend'
    dotnet-dsrouter client-server -tcps 127.0.0.1:9001 -ipcc /tmp/maui-app --verbose debug
    dotnet-trace collect --diagnostic-port /tmp/maui-app --providers Microsoft-Windows-DotNETRuntime:0x1F000080018:5 --output android.nettrace

2. Install `dotnet-pgo`:

    dotnet tool install -g dotnet-pgo --add-source https://pkgs.dev.azure.com/dnceng/public/_packaging/7.0.100-preview.7.22377.5-nonshipping/nuget/v3/index.json --version 7.0.0-preview.7.22375.6

3. Convert to `.mibc`:

    dotnet-pgo create-mibc --trace android.nettrace  --output android.mibc --reference "src/profiled-aot/obj/android/obj/Release/net7.0-android/android-arm64/linked/shrunk/*.dll"

To test this, I made a `dotnet new android` app and added:

    <PropertyGroup>
      <AndroidUseDefaultAotProfile>false</AndroidUseDefaultAotProfile>
    </PropertyGroup>
    <ItemGroup>
      <AndroidMibcProfile Include="android.mibc" />
    </ItemGroup>

~~ Open Questions ~~

1. If you have to use a weird feed for `dotnet-pgo` is that OK? Are we
   not recommending users record their own profiles yet?

2. How do we make an MSBuild target to make recording profiles easier?
   I have not been able to programmatically control `dotnet-trace` in
   a nice way.

For example:

    $ dotnet build -t:BuildAndStartTracing -c Release -r android-arm64

Do stuff in the app, then:

    $ dotnet build -t:FinishTracing

See:

#7087

3. How do we get a list of method names from either the `.nettrace` or
   `.mibc` file? We track the methods in a text file in source control.

See:

https://github.com/xamarin/xamarin-android/blob/9b378e578fa4ff878281da113388f05a4027d484/src/profiled-aot/dotnet.aotprofile.txt
https://github.com/dotnet/maui/blob/08ca198145f27af96e0738853c341dbf33d47804/.nuspec/maui.aotprofile.txt

4. Is `@(AndroidMibcProfile)` the right name for the item group? Or
   should we use `@(AndroidAotProfile)` and just detect the `.mibc`
   file extension? `.mibc` files could pass `mibc-profile=` to the AOT
   compiler.

5. Is it OK if all methods are present in the `.mibc` file? With AOT
   profiles we did:

    aotprofile-tool -sd --filter-module="^(?!android).+" custom.aprof -o dotnet.aotprofile

This removed all methods from `android.dll`, which is the main
assembly of the app recorded.

https://github.com/radekdoulik/aotprofile-tool

~~ TODO ~~

Some things TODO even after the above questions are answered:

- [ ] - ship a `.mibc` file in our workload, setup defaults to use it
- [ ] - update `src/profiled-aot/build.proj` to record `.mibc`
  profiles. I just did them manually right now.
- [ ] - update dotnet/maui after this goes in
  • Loading branch information
jonathanpeppers committed Aug 9, 2022
1 parent 9b378e5 commit 6bbdf58
Show file tree
Hide file tree
Showing 4 changed files with 20 additions and 10 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ They run in a context of an inner build with a single $(RuntimeIdentifier).
RuntimeIdentifier="$(RuntimeIdentifier)"
EnableLLVM="$(EnableLLVM)"
Profiles="@(AndroidAotProfile)"
MibcProfiles="@(AndroidMibcProfile)"
StripLibraries="$(_AndroidAotStripLibraries)">
<Output PropertyName="_Triple" TaskParameter="Triple" />
<Output PropertyName="_ToolPrefix" TaskParameter="ToolPrefix" />
Expand All @@ -84,6 +85,7 @@ They run in a context of an inner build with a single $(RuntimeIdentifier).
<_MonoAOTAssemblies Update="@(_MonoAOTAssemblies)" ProcessArguments="$(AndroidExtraAotOptions)" />
</ItemGroup>
<PropertyGroup>
<_PgoBinaryPath Condition=" '$(_PgoBinaryPath)' == '' ">$(USERPROFILE)\.dotnet\tools\dotnet-pgo</_PgoBinaryPath>
<_MonoAOTCompilerPath>@(MonoAotCrossCompiler->WithMetadataValue('RuntimeIdentifier', '$(RuntimeIdentifier)'))</_MonoAOTCompilerPath>
<_LLVMPath Condition=" '$(EnableLLVM)' == 'true' ">$([System.IO.Path]::GetDirectoryName ('$(_MonoAOTCompilerPath)'))</_LLVMPath>
</PropertyGroup>
Expand All @@ -105,6 +107,8 @@ They run in a context of an inner build with a single $(RuntimeIdentifier).
LLVMPath="$(_LLVMPath)"
LdName="$(_LdName)"
LdFlags="$(_LdFlags)"
PgoBinaryPath="$(_PgoBinaryPath)"
NetTracePath="$(_NetTracePath)"
WorkingDirectory="$(MSBuildProjectDirectory)"
AotArguments="$(AndroidAotAdditionalArguments)">
<Output TaskParameter="CompiledAssemblies" ItemName="_MonoAOTCompiledAssemblies" />
Expand Down
13 changes: 11 additions & 2 deletions src/Xamarin.Android.Build.Tasks/Tasks/GetAotAssemblies.cs
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
using System;
using System.IO;
using System.Text;
using System.Threading.Tasks;
using Microsoft.Android.Build.Tasks;
using Microsoft.Build.Framework;
using Xamarin.Android.Tools;

namespace Xamarin.Android.Tasks
Expand All @@ -13,6 +15,8 @@ public class GetAotAssemblies : GetAotArguments
{
public override string TaskPrefix => "GAOT";

public ITaskItem [] MibcProfiles { get; set; } = Array.Empty<ITaskItem> ();

public override Task RunTaskAsync ()
{
NdkTools ndk = NdkTools.Create (AndroidNdkDirectory, logErrors: EnableLLVM, log: Log);
Expand Down Expand Up @@ -51,9 +55,14 @@ public override Task RunTaskAsync ()
if (Profiles != null && Profiles.Length > 0) {
aotProfiles.Append (",profile-only");
foreach (var p in Profiles) {
var fp = Path.GetFullPath (p.ItemSpec);
aotProfiles.Append (",profile=");
aotProfiles.Append (fp);
aotProfiles.Append (Path.GetFullPath (p.ItemSpec));
}
} else if (MibcProfiles != null && MibcProfiles.Length > 0) {
aotProfiles.Append (",profile-only");
foreach (var p in MibcProfiles) {
aotProfiles.Append (",mibc-profile=");
aotProfiles.Append (Path.GetFullPath (p.ItemSpec));
}
}

Expand Down
3 changes: 2 additions & 1 deletion src/profiled-aot/Directory.Build.props
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@
<PropertyGroup>
<Configuration>Release</Configuration>
<AndroidNeedsInternetPermission>true</AndroidNeedsInternetPermission>
<AndroidEnableAotProfiler>true</AndroidEnableAotProfiler>
<!-- <AndroidEnableAotProfiler>true</AndroidEnableAotProfiler> -->
<AndroidEnableProfiler>true</AndroidEnableProfiler>
<RunAOTCompilation>false</RunAOTCompilation>
</PropertyGroup>

Expand Down
10 changes: 3 additions & 7 deletions src/profiled-aot/Directory.Build.targets
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,14 @@
</PropertyGroup>
<ItemGroup>
<Compile Include="$(MSBuildThisFileDirectory)CommonMethods.cs" />
<AndroidAotProfile Include="custom.aprof" />
<PackageReference Include="Mono.AotProfiler.Android" Version="7.0.0-preview1" />
<!-- <AndroidAotProfile Include="custom.aprof" />
<PackageReference Include="Mono.AotProfiler.Android" Version="7.0.0-preview1" /> -->
</ItemGroup>
<PropertyGroup>
<RecordDependsOn>
Clean;
_ClearSystemProperties;
BuildAndStartAotProfiling;
_Sleep;
FinishAotProfiling;
_StripAppMethods;
_SaveMethodNames;
Run;
</RecordDependsOn>
</PropertyGroup>
<Target Name="Record" DependsOnTargets="$(RecordDependsOn)">
Expand Down

0 comments on commit 6bbdf58

Please sign in to comment.