Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maui Android Size on Disk Integration #60425

Merged
merged 86 commits into from
Oct 19, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
86 commits
Select commit Hold shift + click to select a range
0966751
Removed runs that we don't want during testing.
LoopedBard3 Sep 17, 2021
a16e9d5
First attempt maui build on Mac
LoopedBard3 Sep 28, 2021
ad45d96
Added in rollback file retrieval.
LoopedBard3 Sep 28, 2021
11229f5
Fix apps file name.
LoopedBard3 Sep 28, 2021
5a8f2f0
Fix yaml error about expecting at least on kv pair
LoopedBard3 Sep 28, 2021
89dc235
Skip building the runtime for now.
LoopedBard3 Sep 28, 2021
be935f2
Added gather of more debug information.
LoopedBard3 Sep 28, 2021
8b80175
Fix steps error.
LoopedBard3 Sep 28, 2021
ace4944
Try jobs instead.
LoopedBard3 Sep 28, 2021
3ec88ed
Try again.
LoopedBard3 Sep 28, 2021
e5be8a6
Third times charm.
LoopedBard3 Sep 28, 2021
83054bb
Or not.
LoopedBard3 Sep 28, 2021
1ad683d
Pipelines are hard.
LoopedBard3 Sep 28, 2021
d2c2aa4
Removed a space.
LoopedBard3 Sep 28, 2021
cd74562
Readded jobs arch.
LoopedBard3 Sep 28, 2021
d278a8f
Hopefully fix VM environment.
LoopedBard3 Sep 28, 2021
d1c8e18
Revert back to defaultish perf.yml config.
LoopedBard3 Sep 28, 2021
851b745
Get rid of jobs in the apps.yml.
LoopedBard3 Sep 28, 2021
a0d00b2
Add condition to printing files.
LoopedBard3 Sep 28, 2021
d05da84
Only get files from .dotnet.
LoopedBard3 Sep 28, 2021
ae96166
filter out . and .. files.
LoopedBard3 Sep 28, 2021
7f2339d
Try using a fresh dotnet install instead of the one provided, maybe t…
LoopedBard3 Sep 28, 2021
a86daa4
Removed dotnet install as we don't have perms, added msbuild restore …
LoopedBard3 Sep 29, 2021
f2d097f
Try using the build in dotnet cli to build.
LoopedBard3 Sep 29, 2021
4f822d1
Set more specific dotnet verion to use, and only try to build the mau…
LoopedBard3 Sep 29, 2021
78e7789
Put in correct version.
LoopedBard3 Sep 29, 2021
1e258af
SDK version was not being found, reverting to the autocompleted one.
LoopedBard3 Sep 29, 2021
df041ad
Reverted dotnet calls as they didn't make a difference, split install…
LoopedBard3 Sep 29, 2021
d3fc7a5
dotnet needs a .sh after it.
LoopedBard3 Sep 30, 2021
48f9860
Cannot do xcode-select switch without sudo, could not tell if the and…
LoopedBard3 Sep 30, 2021
132b2d1
Attempt ignoring the higher level props files for MauiSince we don't …
LoopedBard3 Sep 30, 2021
36cde29
Completely isolate the MAUI folder build targets.
LoopedBard3 Sep 30, 2021
eb2d62c
Cleaned up no longer needed and duplicate pipeline tasks.
LoopedBard3 Sep 30, 2021
b596807
Got rid of the android only build portion to test maccatalyst and iOS…
LoopedBard3 Sep 30, 2021
f9724f6
Split each build into it's own step with its own binlog and artifact …
LoopedBard3 Sep 30, 2021
178ebf9
Update the run conditions so each build will run regardless of the la…
LoopedBard3 Sep 30, 2021
14940f1
Modified perf.yml to have a more accurate job description for maui bu…
LoopedBard3 Sep 30, 2021
82a6978
Commented out mac and ios build since they need xcode 13 and the cont…
LoopedBard3 Sep 30, 2021
4ca436a
Cleaned up the perf yml and added the building of the packs. Created …
LoopedBard3 Sep 30, 2021
1451ff1
Added what seems to be an approach to replacing the correct packages …
LoopedBard3 Sep 30, 2021
6afa983
Fix duplicate condition issue.
LoopedBard3 Sep 30, 2021
c251b36
Maybe added build of mono.workloads and search for pack.
LoopedBard3 Sep 30, 2021
a506a12
Added mscordbi to build subset, set install maui to always run as it …
LoopedBard3 Sep 30, 2021
07f8017
Simplified build subsets and attempting to brute force the build for …
LoopedBard3 Oct 1, 2021
412a2eb
Turns out you cannot brute force in the arg types, so instead I am ju…
LoopedBard3 Oct 1, 2021
8b633cd
Fix the location of the find
LoopedBard3 Oct 1, 2021
71e842e
Build failed before other injection seemingly because a pack was miss…
LoopedBard3 Oct 1, 2021
9da47a3
Removed name override, it seems like it may be causing some issues.
LoopedBard3 Oct 1, 2021
4d5ad70
The previous fix was not actually the problem. The android build was …
LoopedBard3 Oct 1, 2021
3707c97
Moved android build to its own script spot for easier failure catchin…
LoopedBard3 Oct 1, 2021
653ba77
Change find to find all potentially correct files that I will eventua…
LoopedBard3 Oct 1, 2021
ba02164
Try to find minimum buid with the ios build, have maui stuff run no m…
LoopedBard3 Oct 1, 2021
c1795d4
Do we really need more than mono? The quest to find the minimum build…
LoopedBard3 Oct 1, 2021
f000d04
Added steps to emulate workload build in main runtime pipeline, this …
LoopedBard3 Oct 4, 2021
cfb0a7d
Fix? job parameter yaml errors.
LoopedBard3 Oct 4, 2021
4b1264c
Had poor indentation.
LoopedBard3 Oct 4, 2021
efe3cdd
Initially forgot to update the depends on for the actual maui build.
LoopedBard3 Oct 4, 2021
fe71cb3
Attempt to fix the depends on parameter.
LoopedBard3 Oct 4, 2021
0dfe604
Fixed app runtime directory location for insertion of new packs.
LoopedBard3 Oct 4, 2021
f533b3c
First iteration of targets that appears to work for overwriting the b…
LoopedBard3 Oct 5, 2021
829e74d
Changed build type to release and updated the extract files command t…
LoopedBard3 Oct 5, 2021
b21750e
Don't delete the downloaded files because if we do, we can't extract …
LoopedBard3 Oct 5, 2021
8c21d09
Fix the renaming to a style that should actually work.
LoopedBard3 Oct 5, 2021
da6d942
Fixed globbing pattern and added a ls for files.
LoopedBard3 Oct 5, 2021
dc6bbb1
Added step to rename nupkg to zip so they will actually be extracted …
LoopedBard3 Oct 5, 2021
1ea6740
Turns out, you have to individually extract these files. So this does…
LoopedBard3 Oct 5, 2021
e2a33a2
removed renaming of folders since extracting the files achieves the s…
LoopedBard3 Oct 5, 2021
453a3ed
Debugging file note accessible error.
LoopedBard3 Oct 6, 2021
e672207
Use chmod properly.. (Don't include emojis yet!)
LoopedBard3 Oct 6, 2021
c807078
Still figuring out how to test properly.
LoopedBard3 Oct 6, 2021
0db5378
Removed file listings and set the apk to be the artifact, tobe tested…
LoopedBard3 Oct 6, 2021
fee5d20
Added first pass of Maui SOD testing.
LoopedBard3 Oct 7, 2021
064b3e7
Update perf setup to match the latest setup for correct branch names.
LoopedBard3 Oct 11, 2021
71c96f0
Setup perf setup to pull from matching perf testing branch.
LoopedBard3 Oct 12, 2021
8b287cf
Updated ls to be dir since the test runs on windows.
LoopedBard3 Oct 12, 2021
3425fff
Add back in the testing commands for first test of actual commands.
LoopedBard3 Oct 12, 2021
3494c41
Removed the dir command and added the extracted size test to see if i…
LoopedBard3 Oct 12, 2021
af2f8cf
Cleaned up the targets and maui-perf.yml files.
LoopedBard3 Oct 12, 2021
5b4798b
See if we can run without the props file to limit unnessary files, etc.
LoopedBard3 Oct 12, 2021
cb5e50c
Cleaned up the maui-apps.yml a touch more
LoopedBard3 Oct 12, 2021
a7e997d
Revert "See if we can run without the props file to limit unnessary f…
LoopedBard3 Oct 12, 2021
5e1a670
Fix typo in the resolved runtime packs architectures.
LoopedBard3 Oct 13, 2021
e467f95
Removed seemingly unused architectures to limit build time and system…
LoopedBard3 Oct 13, 2021
56222f8
Cleanup for final test run before PR.
LoopedBard3 Oct 14, 2021
402bfd8
Finalize the files for code review and merge.
LoopedBard3 Oct 14, 2021
d91fa97
Add back a space.
LoopedBard3 Oct 14, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 49 additions & 1 deletion eng/pipelines/coreclr/perf.yml
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,7 @@ jobs:
platforms:
- Linux_x64

# run mono android scenarios
# run mono and maui android scenarios
- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/coreclr/templates/perf-job.yml
Expand Down Expand Up @@ -484,3 +484,51 @@ jobs:
additionalSetupParameters: '--latestdotnet'
logicalmachine: 'perftiger'

# build maui workload packs
- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/common/global-build-job.yml
buildConfig: release
runtimeFlavor: mono
platforms:
- Android_x86
- Android_arm64
jobParameters:
buildArgs: -s mono+libs+host+packs -c $(_BuildConfig)
nameSuffix: Maui_Packs_Mono
isOfficialBuild: false
extraStepsTemplate: /eng/pipelines/common/upload-intermediate-artifacts-step.yml
extraStepsParameters:
name: MonoRuntimePacks
# Eventual platforms to add
# - MacCatalyst_x64
# - MacCatalyst_arm64
# - iOS_arm
# - iOS_arm64
# - OSX_x64
# - OSX_arm64

# build maui app
- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/common/global-build-job.yml
buildConfig: release
runtimeFlavor: mono
platforms:
- iOS_arm64
jobParameters:
dependsOn:
- Build_Android_arm64_release_Maui_Packs_Mono
- Build_Android_x86_release_Maui_Packs_Mono
buildArgs: -s mono -c $(_BuildConfig)
nameSuffix: MACiOSAndroidMaui
isOfficialBuild: false
extraStepsTemplate: /eng/pipelines/coreclr/templates/build-perf-maui-apps.yml
extraStepsParameters:
rootFolder: '$(Build.SourcesDirectory)/artifacts/'
includeRootFolder: true
displayName: MAC, iOS, and Android Maui Artifacts
artifactName: MACiOSAndroidMauiArm
archiveExtension: '.tar.gz'
archiveType: tar
tarCompression: gz
131 changes: 131 additions & 0 deletions eng/pipelines/coreclr/templates/build-perf-maui-apps.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
parameters:
osGroup: ''
osSubgroup: ''
archType: ''
buildConfig: ''
runtimeFlavor: ''
helixQueues: ''
targetRid: ''
nameSuffix: ''
platform: ''
shouldContinueOnError: ''
rootFolder: ''
includeRootFolder: ''
displayName: ''
artifactName: ''
archiveExtension: ''
archiveType: ''
tarCompression: ''


steps:
- task: DownloadPipelineArtifact@2
displayName: Download runtime packages
inputs:
artifact: 'IntermediateArtifacts'
path: $(Build.SourcesDirectory)/MauiTesting/ArtifactPacks
patterns: |
IntermediateArtifacts/MonoRuntimePacks/Shipping/Microsoft.NETCore.App.Runtime.Mono.android-!(*.symbols).nupkg

# Other artifacts to include once they are being built
# IntermediateArtifacts/MonoRuntimePacks/Shipping/Microsoft.NETCore.App.Runtime.Mono.ios-*.nupkg
# IntermediateArtifacts/MonoRuntimePacks/Shipping/Microsoft.NETCore.App.Runtime.Mono.iossimulator-*.nupkg
# IntermediateArtifacts/MonoRuntimePacks/Shipping/Microsoft.NETCore.App.Runtime.Mono.maccatalyst-*.nupkg

- task: CopyFiles@2
displayName: Flatten packages
inputs:
sourceFolder: $(Build.SourcesDirectory)/MauiTesting/ArtifactPacks
contents: '*/Shipping/*.nupkg'
cleanTargetFolder: false
targetFolder: $(Build.SourcesDirectory)/MauiTesting/ArtifactPacks
flattenFolders: true

- script: |
for file in *.nupkg
do
mv -v "$file" "${file%.nupkg}.zip"
done
displayName: Change nupkgs to zips
workingDirectory: $(Build.SourcesDirectory)/MauiTesting/ArtifactPacks


# #Unzip the nuget packages to make the actual runtimes accessible
- task: ExtractFiles@1
displayName: Extract android-arm64 runtime
inputs:
archiveFilePatterns: $(Build.SourcesDirectory)/MauiTesting/ArtifactPacks/Microsoft.NETCore.App.Runtime.Mono.android-arm64.*.zip
destinationFolder: $(Build.SourcesDirectory)/MauiTesting/ArtifactPacks/Microsoft.NETCore.App.Runtime.Mono.android-arm64
overwriteExistingFiles: true
cleanDestinationFolder: false
- task: ExtractFiles@1
displayName: Extract android-x86 runtime
inputs:
archiveFilePatterns: $(Build.SourcesDirectory)/MauiTesting/ArtifactPacks/Microsoft.NETCore.App.Runtime.Mono.android-x86.*.zip
destinationFolder: $(Build.SourcesDirectory)/MauiTesting/ArtifactPacks/Microsoft.NETCore.App.Runtime.Mono.android-x86
overwriteExistingFiles: true
cleanDestinationFolder: false

- script: |
curl -o ./rollback.json 'maui.blob.core.windows.net/metadata/rollbacks/main.json'
./dotnet.sh workload update --from-rollback-file ./rollback.json
./dotnet.sh workload install maui --skip-manifest-update
displayName: Install MAUI workload
workingDirectory: $(Build.SourcesDirectory)

- script: |
./dotnet.sh new maui -n MauiTesting
cd MauiTesting
cp $(Build.SourcesDirectory)/src/maui/MauiScenario.props ./Directory.Build.props
cp $(Build.SourcesDirectory)/src/maui/MauiScenario.targets ./Directory.Build.targets
displayName: Setup MAUI Project
workingDirectory: $(Build.SourcesDirectory)

- script: |
chmod -R a+r .
../dotnet.sh publish -bl:MauiAndroid.binlog -f net6.0-android -c Release
mv ./bin/Release/net6.0-android/com.companyname.MauiTesting-Signed.apk ./MauiAndroidDefault.apk
displayName: Build MAUI Android
workingDirectory: $(Build.SourcesDirectory)/MauiTesting

- task: PublishBuildArtifacts@1
displayName: 'Publish MauiAndroid binlog'
inputs:
pathtoPublish: $(Build.SourcesDirectory)/MauiTesting/MauiAndroid.binlog
artifactName: ${{ parameters.artifactName }}

- template: /eng/pipelines/common/upload-artifact-step.yml
parameters:
rootFolder: $(Build.SourcesDirectory)/MauiTesting/MauiAndroidDefault.apk
includeRootFolder: true
displayName: Maui Android App
artifactName: MauiAndroidApp
archiveExtension: '.tar.gz'
archiveType: tar
tarCompression: gz

- script: rm -r -f ./bin
workingDirectory: $(Build.SourcesDirectory)/MauiTesting
displayName: Clean bin directory
condition: succeededOrFailed()

- template: /eng/pipelines/common/upload-artifact-step.yml
parameters:
osGroup: ${{ parameters.osGroup }}
osSubgroup: ${{ parameters.osSubgroup }}
archType: ${{ parameters.archType }}
buildConfig: ${{ parameters.buildConfig }}
runtimeFlavor: ${{ parameters.runtimeFlavor }}
helixQueues: ${{ parameters.helixQueues }}
targetRid: ${{ parameters.targetRid }}
nameSuffix: ${{ parameters.nameSuffix }}
platform: ${{ parameters.platform }}
shouldContinueOnError: ${{ parameters.shouldContinueOnError }}
rootFolder: ${{ parameters.rootFolder }}
includeRootFolder: ${{ parameters.includeRootFolder }}
displayName: ${{ parameters.displayName }}
artifactName: ${{ parameters.artifactName }}
archiveExtension: ${{ parameters.archiveExtension }}
archiveType: ${{ parameters.archiveType }}
tarCompression: ${{ parameters.tarCompression }}

9 changes: 9 additions & 0 deletions eng/pipelines/coreclr/templates/perf-job.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ jobs:
- ${{ format('build_{0}{1}_{2}_{3}_{4}', parameters.osGroup, parameters.osSubgroup, parameters.archType, parameters.buildConfig, parameters.codeGenType) }}
- ${{ if eq(parameters.runtimeType, 'AndroidMono')}}:
- ${{ 'build_Android_arm64_release_AndroidMono' }}
- ${{ 'Build_iOS_arm64_release_MACiOSAndroidMaui' }}
- ${{ if eq(parameters.runtimeType, 'iOSMono')}}:
- ${{ 'build_iOS_arm64_release_iOSMono' }}

Expand Down Expand Up @@ -157,6 +158,14 @@ jobs:
artifactFileName: 'AndroidMonoarm64.tar.gz'
artifactName: 'AndroidMonoarm64'
displayName: 'Mono Android runtime'
- template: /eng/pipelines/common/download-artifact-step.yml
parameters:
unpackFolder: $(Build.SourcesDirectory)
cleanUnpackFolder: false
artifactFileName: 'MauiAndroidApp.tar.gz'
artifactName: 'MauiAndroidApp'
displayName: 'Maui Android App'


# Download iOSMono tests
- ${{ if eq(parameters.runtimeType, 'iOSMono') }}:
Expand Down
12 changes: 12 additions & 0 deletions eng/testing/performance/android_scenarios.proj
Original file line number Diff line number Diff line change
Expand Up @@ -30,5 +30,17 @@
<Command>$(Python) test.py sod --scenario-name &quot;%(Identity)&quot;</Command>
<PostCommands>$(Python) post.py</PostCommands>
</HelixWorkItem>
<HelixWorkItem Include="SOD - Maui Android APK Size">
<PayloadDirectory>$(WorkItemDirectory)</PayloadDirectory>
<PreCommands>cd $(ScenarioDirectory)mauiandroid;copy %HELIX_CORRELATION_PAYLOAD%\MauiAndroidDefault.apk .; $(Python) pre.py</PreCommands>
<Command>$(Python) test.py sod --scenario-name &quot;%(Identity)&quot;</Command>
<PostCommands>$(Python) post.py</PostCommands>
</HelixWorkItem>
<HelixWorkItem Include="SOD - Maui Android Extracted Size">
<PayloadDirectory>$(WorkItemDirectory)</PayloadDirectory>
<PreCommands>cd $(ScenarioDirectory)mauiandroid;copy %HELIX_CORRELATION_PAYLOAD%\MauiAndroidDefault.apk .; $(Python) pre.py --unzip</PreCommands>
<Command>$(Python) test.py sod --scenario-name &quot;%(Identity)&quot;</Command>
<PostCommands>$(Python) post.py</PostCommands>
</HelixWorkItem>
</ItemGroup>
</Project>
1 change: 1 addition & 0 deletions eng/testing/performance/performance-setup.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,7 @@ if ($AndroidMono) {
mkdir $WorkItemDirectory
}
Copy-Item -path "$SourceDirectory\artifacts\bin\AndroidSampleApp\arm64\Release\android-arm64\publish\apk\bin\HelloAndroid.apk" $PayloadDirectory
Copy-Item -path "$SourceDirectory\MauiAndroidDefault.apk" $PayloadDirectory
$SetupArguments = $SetupArguments -replace $Architecture, 'arm64'
}

Expand Down
3 changes: 3 additions & 0 deletions src/maui/MauiScenario.props
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
<Project>
<!-- Only here so that the default Directory.Build.props will not be used. -->
</Project>
19 changes: 19 additions & 0 deletions src/maui/MauiScenario.targets
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
<Project>
<Target Name="PrintRuntimePackLocation" AfterTargets="UpdateTargetingAndRuntimePack">
<Message Text="Used runtime pack: %(ResolvedRuntimePack.PackageDirectory)" Importance="high" />
</Target>

<!-- Use local targeting pack for NetCoreAppCurrent. -->
<Target Name="UpdateTargetingAndRuntimePack"
AfterTargets="ResolveFrameworkReferences">
<PropertyGroup>
<MicrosoftNetCoreAppRuntimePackDir>./ArtifactPacks</MicrosoftNetCoreAppRuntimePackDir>
</PropertyGroup>
<ItemGroup>
<ResolvedRuntimePack PackageDirectory="$(MicrosoftNetCoreAppRuntimePackDir)/Microsoft.NETCore.App.Runtime.Mono.android-arm64"
Condition="'%(ResolvedRuntimePack.FrameworkName)' == 'Microsoft.NETCore.App' and '%(ResolvedRuntimePack.RuntimeIdentifier)' == 'android-arm64'" />
<ResolvedRuntimePack PackageDirectory="$(MicrosoftNetCoreAppRuntimePackDir)/Microsoft.NETCore.App.Runtime.Mono.android-x86"
Condition="'%(ResolvedRuntimePack.FrameworkName)' == 'Microsoft.NETCore.App' and '%(ResolvedRuntimePack.RuntimeIdentifier)' == 'android-x86'" />
</ItemGroup>
</Target>
</Project>