FSC: Consider enabling background JIT with non-dynamic PGO for faster compilation

[safesparrow]

Context

I noticed that JIT takes a considerable amount of time in compilation, and the fraction of time it takes is especially high for small projects.

I experimented with using non-dynamic PGO for JITting to allow JITting to happen in the background, before its results are needed.

Tests

I tested it against an empty project from the dotnet template (just one file). The PGO-enabled compilation was ~300ms faster (~0.95s -> ~0.65s). I generated the profile by running compilation on the same test project a few times, and measured timings then.

Here is a profile showing JIT work happening without PGO:

It shows 1200ms of JIT on the main thread, and similar amount on other threads (mostly blocking actual work).

Here is a profile showing JIT work happening with PGO:

It shows 835ms of JIT on the main thread, ~700ms on other work threads, and 1250ms on a dedicated JITting thread (CLR worker).

Questions

• Am I missing something? Is non-dynamic PGO being used already somehow?
• Are there any major drawbacks to using this?
• If enabled, how would the profile be deployed/generated on the end-user's machine? Would hiding it behind a feature flag/env variable make sense?
• Can SDK ship no profile and let the user's first build generate one? We could use profile names based on the name of the project being compiled. This means no deployment changes are needed, and it can be fully controlled by the enduser
• I know dynamic PGO exists in NET6, but I don't think that's needed for the compilation process.
• I'm aware background compiler service might become a reality at some point, making JIT irrelevant, but this solution seems so cheap that it should be worth considering anyway.

[vzarytovskii]

Might be superceded by #13328

@EgorBo might know answers to some of the questions

[safesparrow]

This feels like a low-cost high value performance improvement. Are there plans to pick it up again? @vzarytovskii

[EgorBo]

I noticed that JIT takes a considerable amount of time in compilation, and the fraction of time it takes is especially high for small projects.

The main idea is to rely on R2R to save time for startup, #13328 should improve R2R coverage. What exactly did you measure - the dotnet build command for a hello world?

[safesparrow]

What exactly did you measure - the dotnet build command for a hello world?

1. Create template with dotnet new classlib --name library -lang F#
2. Modify the fsproj to use a locally-built compiler:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net7.0</TargetFramework>
    <GenerateDocumentationFile>true</GenerateDocumentationFile>
    <DisableAutoSetFscCompilerPath>false</DisableAutoSetFscCompilerPath>
    <DotnetFscCompilerPath>c:/projekty/fsharp/fsharp_main/artifacts/bin/fsc/Release/net7.0/win-x64/publish/fsc.dll</DotnetFscCompilerPath>
    <FSharpPreferNetFrameworkTools>False</FSharpPreferNetFrameworkTools>
    <FSharpPrefer64BitTools>True</FSharpPrefer64BitTools>
  </PropertyGroup>

  <ItemGroup>
    <Compile Include="Library.fs" />
  </ItemGroup>

</Project>

3. Publish a ReadyToRun Release compiler with

dotnet publish .\src\fsc\fscProject\fsc.fsproj -c Release -r win-x64 -p:PublishReadyToRun=true -f net7.0 --no-self-contained

4. Compile the library 10 times (manually), disabling the profile with a custom env variable:

$env:FSC_Profile=0
rm ./obj/Debug/ -r ; rm ./bin/Debug/ -r ; dotnet build --no-restore --no-dependencies /clp:PerformanceSummary

The Fsc task's times are 850ms - 930ms

5. Now run the same with profile optimization, 10 times:

$env:FSC_Profile=1
rm ./obj/Debug/ -r ; rm ./bin/Debug/ -r ; dotnet build --no-restore --no-dependencies /clp:PerformanceSummary

Warmup run (no existing profile): 965ms
Next 10 runs: Fastest: 583ms, slowest: 658ms

[EgorBo]

I suspect it highlights the gap in R2R coverage rather than motivates to enable profile-driven Multicore JIT. Ideally, F# compiler should jit nothing to compile an app

[safesparrow]

If making it do no JIT is as easy as using background JIT then sure. But is it really, given that it's been looked at a year ago but didn't get any traction?

Personally I would be very happy to use opt-in background JIT right now, instead of waiting.
It doesn't seem controversial with an opt-in mechanism.

[EgorBo]

If making it do no JIT is as easy as using background JIT then sure. But is it really, given that it's been looked at a year ago but didn't get any traction?

Personally I would be very happy to use opt-in background JIT right now, instead of waiting. It doesn't seem controversial with an opt-in mechanism.

You need to measure its impact on large codebases too first, e.g. .NET 8.0 has dynamic PGO enabled by default that might help for long-running processes, but Multicore JIT won't use it afair. Although, I know very little about it and how well tested it is maybe @jkotas knows better?

[vzarytovskii]

This feels like a low-cost high value performance improvement. Are there plans to pick it up again? @vzarytovskii

Yeah, it's currently in progress, but unfortunately involves a lot more infrastructure changes than just having some flag enabled somewhere.

[safesparrow]

@EgorBo you're right that I should do more testing. Although I'd argue that reducing compilation time of tiny projects would go a long way encouraging people to try fsharp.

Compilation is not really a long running process though. It would be if we used compiler service but we don't.

@vzarytovskii The in-progress solution not being trivial is my point. The flag I tested is 3 lines of code and requires no changes in the deployment. The profile can be generated by the user in a temp directory, with different profile for different project name being compiled.

If it was available right now, I would definitely want to try it.

[jkotas]

I know very little about it and how well tested it is maybe @jkotas knows better?

Multicore JIT is supported feature. If you run into functional problems with it, we will investigate them. It is used by Powershell.

[vzarytovskii]

The flag I tested is 3 lines of code and requires no changes in the deployment.

It only looks like it :) We need to make changes to several places, including (probably) our signed build (both SDK and VS), dotnet/dotnet, source build, maybe something else (because we don't "maintain" flags for publishing).

MIBC might be the easiest solution here (I have added almost everything which is needed, so it's already in progress).

[psfinaki]

Hey @safesparrow - I am looking through the startup perf related issues and trying to get my head around them.

Regarding this particular ticket, does it still hold?

• this was opened in .NET 7 times and looks like the some jit-related defaults have changed since
• I didn't entirely understand the $env:FSC_Profile=0 thing. The article talks about apps whereas you mention a library here, so not sure how you were creating this profile
• as a result, i couldn't reproduce the perf improvement here

I am just starting with this topic so might be horribly missing something. I will be primarily looking into MIBC but just wanted to understand if your particular suggestion here is still relevant.

[vzarytovskii]

This item should be closed in favour using mibc profiles runtime provides for us.

[safesparrow]

The FSC_Profile was an env var that my local version of the compiler exe used to decide if it should generate/read the JIT profile or not at startup.

I haven't looked into this in a long time and won't have much time to look at it again soon.

I'd be happy if any low hanging fruit were implemented - unless those are already in the latest compiler.

[psfinaki]

Thanks for the clarification. Yeah, we're looking into this, stay tuned :)