Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Find a way to embed project.assets.json in .binlog #3529

Closed
KirillOsenkov opened this issue Jul 19, 2018 · 6 comments · Fixed by dotnet/sdk#16840
Closed

Find a way to embed project.assets.json in .binlog #3529

KirillOsenkov opened this issue Jul 19, 2018 · 6 comments · Fixed by dotnet/sdk#16840
Assignees
Labels
Area: Debuggability Issues impacting the diagnosability of builds, including logging and clearer error messages. Area: Logging triaged

Comments

@KirillOsenkov
Copy link
Member

It would be helpful to include the project.assets.json file from the obj directory in the .binlog file archive.

Not sure what's the best way to do it as it's not technically an MSBuild file.

@KirillOsenkov KirillOsenkov added the Area: Debuggability Issues impacting the diagnosability of builds, including logging and clearer error messages. label Jul 19, 2018
@rainersigwald
Copy link
Member

What scenarios aren't well-served by just looking at the results of the asset-file-reading task?

Maybe some kind of advisory API from within a task: "it'd be interesting to capture this file"?

IBuildEngine6.LogFileContents(string path)

(Someday we're going to get into double digits on that interface and it will be something to behold.)

We wouldn't want folks to abuse this for everything (if you need to debug contents of source files, build locally), but for build-graph-altering things like the assets file it could be handy.

@KirillOsenkov
Copy link
Member Author

Here's an example where I needed the contents of project.assets.json:
NuGet/Home#7132

@KirillOsenkov
Copy link
Member Author

A naïve way to do that that works well is just add this to BinaryLogger.CollectImports():

                if (projectArgs.Items is IEnumerable items)
                {
                    foreach (DictionaryEntry item in items)
                    {
                        if (item.Key is string itemName && itemName == "IncludeInBinlog" && item.Value is ITaskItem taskItem)
                        {
                            projectImportsCollector.AddFile(taskItem.ItemSpec);
                        }
                    }
                }

                if (projectArgs.Properties is IEnumerable properties)
                {
                    foreach (DictionaryEntry property in properties)
                    {
                        if (property.Key is string name && name == "ProjectAssetsFile" && property.Value is string projectAssetsFile)
                        {
                            projectImportsCollector.AddFile(projectAssetsFile);
                        }
                    }
                }

@KirillOsenkov
Copy link
Member Author

Suggestions welcome on how to name the item IncludeInBinlog that the binary logger will recognize as special.

Then you could just do

<ItemGroup>
    <IncludeInBinlog Include="CustomFile.txt" />
</ItemGroup>

to embed the file in binlog.

Embedding all project.assets.json roughly doubles the size of the files archive, from 450K to 900K. Feels like it's worth it though because project.assets.json contain valuable information not available elsewhere, e.g. why was a certain file chosen to be CopyLocal.

@KirillOsenkov
Copy link
Member Author

It's unclear when to embed the file. ProjectStarted or ProjectEvaluationFinished may be too early as the file may not exist yet. But I guess in the majority of cases when a project is being built the Restore already happened.

TargetFinished for Restore target could be another location.

@KirillOsenkov
Copy link
Member Author

I feel like I should first fix #5316 so we can better enumerate items after evaluation.

KirillOsenkov added a commit that referenced this issue Apr 10, 2021
It would be very useful to embed more than just the project files into the binlog. For example, project.assets.json is a common request: #3529. global.json or nuget.config would be other examples. Some users may want to embed all *.cs files as well.

Add a special EmbedInBinlog item type recognized by BuildEventArgsWriter. If after evaluation the items contain EmbedInBinlog, all such items will be logged if the evaluated include points to a full file path that exists on disk. A given file can be mentioned more than once, but will only be embedded once. If a file doesn't exist or is empty, it will not be embedded. If the item is added from a target (after evaluation is done), it will not be embedded.

Checking for EmbedInBinlog is done in BuildEventArgsWriter to reuse the complicated and performance-optimized logic of iterating over all items.
KirillOsenkov added a commit to KirillOsenkov/sdk that referenced this issue Apr 11, 2021
We're thinking of adding a way to embed arbitrary files in the binlog.
See dotnet/msbuild#6339

project.assets.json is commonly requested to be embedded. This would fix dotnet/msbuild#3529

Binlog size does increase from 3.5 MB -> 5 MB, so we'll need to think perhaps about making it off by default.
@AR-May AR-May added the triaged label Feb 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: Debuggability Issues impacting the diagnosability of builds, including logging and clearer error messages. Area: Logging triaged
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants