Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement PGO in pipelines for AMD64 architecture; supply training test scenarios #10071

Merged
42 commits merged into from
May 13, 2021

Conversation

miniksa
Copy link
Member

@miniksa miniksa commented May 10, 2021

Implement PGO in pipelines for AMD64 architecture; supply training test scenarios

References

PR Checklist

Detailed Description of the Pull Request / Additional comments

  • Creates a new pipeline run for creating instrumented binaries for Profile Guided Optimization (PGO).
  • Creates a new suite of UIA tests on the full Windows Terminal app to run PGO training scenarios on instrumented binaries (and incidentally can be used to write other UIA tests later for the full Terminal app.)
  • Creates a new NuGet artifact to store trained PGO databases (PGD files) at Microsoft.Internal.Windows.Terminal.PGODatabase
  • Creates a new NuGet artifact to supply large-scale test content for automated tests at Microsoft.Internal.Windows.Terminal.TestContent
  • Adjusts the release pipeline to run binaries in PGO optimized mode where content from PGO databases is leveraged at link time to optimize the final release build

The following binaries are trained:

  • OpenConsole.exe
  • WindowsTerminal.exe
  • TerminalApp.dll
  • TerminalConnection.dll
  • Microsoft.Terminal.Control.dll
  • Microsoft.Terminal.Remoting.dll
  • Microsoft.Terminal.Settings.Editor.dll
  • Microsoft.Terminal.Settings.Model.dll

In the future, adding <PgoTarget>true</PgoTarget> to a new vcxproj file will automatically enroll the DLL/EXE for PGO instrumentation and optimization going forward.

Two training test scenarios are implemented:

  • Smoke test the Terminal by just opening it and typing a bit of text then exiting. (Should help focus on the standard launch path.)
  • Optimize bulk text output by launching terminal, outputting big.txt, then exiting.

Additional scenarios can be contributed to the WindowsTerminal_UIATests project with the [TestProperty("IsPGO", "true")] annotation to add them to the suite of scenarios for PGO.

NOTE: There are currently no weights applied to the various test scenarios. We will revisit that in the future when/if necessary.

Validation Steps Performed

  • - Training run completed at https://dev.azure.com/ms/terminal/_build?definitionId=492&_a=summary
  • - Optimization run completed locally (by forcing PGOBuildMode to Optimize on my local machine, manually retrieving the databases with NuGet, and building).
  • - Validated locally that x86 and ARM64 do not get trained and automatically skip optimization as databases are not present for them.
  • - Smoke tested optimized binary versus latest releases. big.txt output through CMD is ~11-12seconds prior to PGO and just over 8 seconds with PGO.

miniksa added 30 commits May 5, 2021 13:18
…ct. Add smoke test to project to just launch the app.
This reverts commit 3616c2d.
…nctions for url query and downloads as obviously MUX had a problem with it being sporatic too.
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-Build Issues pertaining to the build system, CI, infrastructure, meta Area-Performance Performance-related issue AutoMerge Marked for automatic merge by the bot when requirements are met Issue-Feature Complex enough to require an in depth planning process and actual budgeted, scheduled work. Product-Meta The product is the management of the products.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Apply PGO to Binaries Terminal: UIA test harness
4 participants