Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run TransparentCompiler unit tests with local response files. #16609

Merged
merged 6 commits into from
Jan 30, 2024

Conversation

nojaf
Copy link
Contributor

@nojaf nojaf commented Jan 30, 2024

Description

To experiment with the new Transparent Compiler on local projects, I've implemented infrastructure capable of handling a response file from a local project. This approach mirrors the tests in CompilationFromCmdlineArgsTests, offering valuable insights for local testing.

How to generate a response file?

Poor man's response file

Just run your regular dotnet build -v:n with Verbosity set to normal, wait until you see CoreCompile and grab the compiler arguments.

Via script

The scrape.fsx script does a build of a project and extracts it from the bin log file.

Via Telplin

My Telplin tool has a bit of a hidden feature that it can save the response file without generating any signatures at all.
dotnet tool install -g telplin and then telplin --only-record MyProject.fsproj

Checklist

  • Test cases added

  • Performance benchmarks added in case of performance changes

  • Release notes entry updated:

    Please make sure to add an entry with short succinct description of the change as well as link to this pull request to the respective release notes file, if applicable.

    Release notes files:

    • If anything under src/Compiler has been changed, please make sure to make an entry in docs/release-notes/.FSharp.Compiler.Service/<version>.md, where <version> is usually "highest" one, e.g. 42.8.200
    • If language feature was added (i.e. LanguageFeatures.fsi was changed), please add it to docs/releae-notes/.Language/preview.md
    • If a change to FSharp.Core was made, please make sure to edit docs/release-notes/.FSharp.Core/<version>.md where version is "highest" one, e.g. 8.0.200.

    Information about the release notes entries format can be found in the documentation.
    Example:

    If you believe that release notes are not necessary for this PR, please add NO_RELEASE_NOTES label to the pull request.

@nojaf nojaf requested a review from a team as a code owner January 30, 2024 08:48
@nojaf nojaf requested a review from 0101 January 30, 2024 08:48
@nojaf
Copy link
Contributor Author

nojaf commented Jan 30, 2024

FYI @dawedawe @safesparrow

Copy link
Contributor

✅ No release notes required

Copy link
Contributor

@0101 0101 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great, thanks!

It will probably need a refactoring at some point, since it's not just synthetic projects anymore. Maybe ProjectWorkflowBuilder could just work with FSharpProjectSnapshot eventually...

@nojaf
Copy link
Contributor Author

nojaf commented Jan 30, 2024

It will probably need a refactoring at some point, since it's not just synthetic projects anymore. Maybe ProjectWorkflowBuilder could just work with FSharpProjectSnapshot eventually...

Yes, it is indeed becoming a bit of a hybrid between generated files and real files.

@0101 0101 merged commit 33e8be6 into dotnet:main Jan 30, 2024
28 checks passed
@nojaf
Copy link
Contributor Author

nojaf commented Jan 31, 2024

If anyone ever wants some inspiration how to apply this in a benchmark:

       
[<MemoryDiagnoser>]
[<ThreadingDiagnoser>]
[<SimpleJob(warmupCount=1,iterationCount=4)>]
[<BenchmarkCategory(LongCategory)>]
type RspFileTransparentCompilerBenchmark() =
    let mutable benchmark : ProjectWorkflowBuilder = Unchecked.defaultof<_>
    let responseFile = FileInfo @"C:\Users\nojaf\Projects\g-research-fsharp-analyzers\src\FSharp.Analyzers\FSharp.Analyzers.rsp"
    let syntheticProject = mkSyntheticProjectForResponseFile responseFile

    [<ParamsAllValues>]
    member val UseTransparentCompiler = true with get,set

    [<GlobalSetup>]
    member this.Setup() =
        benchmark <-
            ProjectWorkflowBuilder(
            syntheticProject,
            isExistingProject = true,
            useTransparentCompiler = this.UseTransparentCompiler,
            runTimeout = 15_000).CreateBenchmarkBuilder()

    [<Benchmark>]
    member this.TypeCheckLastFile() =

        use _ = Activity.start "Benchmark" [
            "UseTransparentCompiler", this.UseTransparentCompiler.ToString()
        ]

        let lastFile =
            syntheticProject.SourceFiles
            |> List.tryLast
            |> Option.map (fun sf -> sf.Id)

        match lastFile with
        | None -> failwithf "Last file of project could not be found"
        | Some lastFile ->

        benchmark {
            clearCache
            checkFile lastFile expectOk
        }

I'm not sure this exact thing is useful to contribute but at least there is a trace of it now on the internet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

4 participants