Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support analysis of profiles built with skymeld enabled #97

Closed
lrgoldstein opened this issue Feb 9, 2023 · 1 comment
Closed

Support analysis of profiles built with skymeld enabled #97

lrgoldstein opened this issue Feb 9, 2023 · 1 comment
Assignees
@saraadams
Labels
type/feat Suggests new features.

Comments

@lrgoldstein
Copy link
Contributor

Problem

Bazel's Project Skymeld interleaves the loading/analysis and execution phases. bazel_invocation_analyzer currently depends on separate build markers for those phases in its analysis. This project replaces those 3 build phase markers with a new "Load, analyze dependencies and build artifacts" marker, which breaks some of our analyses.

Support needs to be added for this new build marker in our analysis.

Additional information

Sample build marker entry:
{"cat":"build phase marker","name":"Load, analyze dependencies and build artifacts","ph":"i","ts":1703090,"pid":1,"tid":1930},
@lrgoldstein lrgoldstein added the type/feat Suggests new features. label Feb 9, 2023
@lrgoldstein lrgoldstein mentioned this issue Feb 9, 2023
Closed
saraadams added a commit that referenced this issue Nov 23, 2023
@saraadams
[DataProvider] Add SkymeldUsed Datum and provide it
3190319 
As a first step to supporting Bazel profiles where Skymeld was used,
add a `Datum` that indices whether the profile looks like it was generated
while using Skymeld.

Contributes to #97

Signed-off-by: Sara Adams <[email protected]>
saraadams added a commit that referenced this issue Nov 23, 2023
@saraadams
[DataProvider] Add SkymeldUsed Datum and provide it
8a23e14 
As a first step to supporting Bazel profiles where Skymeld was used,
add a `Datum` that indices whether the profile looks like it was generated
while using Skymeld.

Contributes to #97

Signed-off-by: Sara Adams <[email protected]>
@saraadams saraadams mentioned this issue Nov 23, 2023
Merged
@saraadams saraadams self-assigned this Nov 23, 2023
@saraadams
Copy link
Collaborator

After sifting through a bunch of profiles, I believe a good strategy might be to check for the first event with category action processing to determine when the Load, analyze dependencies and build artifacts phase no longer is just busy with analysis.
I'm not sure whether we might have to exclude the event BazelWorkspaceStatusAction stable-status.txt, as it looks like it might not need a completed analysis phase. However, for the examples I studied, it looked like it was not executed in the traditional analysis phase, but at the start of the traditional execution phase. In that, it might also be a helpful marker of the transition from "only loading and analysis" to "also execution interleaved".

saraadams added a commit that referenced this issue Nov 24, 2023
@saraadams
[Skymeld] Adjust DataProviders and SuggestProviders for Skymeld
9f7e39e 
This change adds support for analysing profiles generated when using Skymeld.
For this, the interlaced analysis and execution phase is split into two parts:

1. Analysis only, no execution interlaced.
2. Execution has started. analysis and execution may be interlaced.

This is done by checking for the first observed `action processing` event.
The distinction is relevant, because it helps us determine how many cores
are available on the machine that ran the Bazel client, as well has how many
cores are used during execution. The later may be higher than the prior,
e.g. when remote execution is used.

* Update `SkymeldUsed` to include the combined analysis and execution phase,
    as well as optionally the phase at which execution likely starts.
* Update `EstimatedCoresDataProvider` to estimate cores when Skymeld is used.
* Update `CriticalPathNotDominantSuggestionProvider` when Skymeld is used,
    if the information when execution started is available.

Contributes to #97.

Signed-off-by: Sara Adams <[email protected]>
@saraadams saraadams mentioned this issue Nov 24, 2023
Merged
saraadams added a commit that referenced this issue Nov 24, 2023
@saraadams
[DataProvider] Add SkymeldUsed Datum and provide it (#116)
f34ee71 
As a first step to supporting Bazel profiles where Skymeld was used, add
a `Datum` that indices whether the profile looks like it was generated
while using Skymeld.

Contributes to #91
Contributes to #97

---------

Signed-off-by: Sara Adams <[email protected]>
saraadams added a commit that referenced this issue Nov 24, 2023
@saraadams
[Skymeld] Adjust DataProviders and SuggestProviders for Skymeld
c673354 
This change adds support for analysing profiles generated when using Skymeld.
For this, the interlaced analysis and execution phase is split into two parts:

1. Analysis only, no execution interlaced.
2. Execution has started. analysis and execution may be interlaced.

This is done by checking for the first observed `action processing` event.
The distinction is relevant, because it helps us determine how many cores
are available on the machine that ran the Bazel client, as well has how many
cores are used during execution. The later may be higher than the prior,
e.g. when remote execution is used.

* Update `SkymeldUsed` to include the combined analysis and execution phase,
    as well as optionally the phase at which execution likely starts.
* Update `EstimatedCoresDataProvider` to estimate cores when Skymeld is used.
* Update `CriticalPathNotDominantSuggestionProvider` when Skymeld is used,
    if the information when execution started is available.

Contributes to #97.

Signed-off-by: Sara Adams <[email protected]>
saraadams added a commit that referenced this issue Nov 27, 2023
@saraadams
[Skymeld] Adjust DataProviders and SuggestProviders for Skymeld (#…
a4fa1e4 
…117)

This change adds support for analysing profiles generated when using
Skymeld. For this, the interlaced analysis and execution phase is split
into two parts:

1. Analysis only, no execution interlaced.
2. Execution has started. analysis and execution may be interlaced.

This is done by checking for the first observed `action processing`
event. The distinction is relevant, because it helps us determine how
many cores are available on the machine that ran the Bazel client, as
well has how many cores are used during execution. The later may be
higher than the prior, e.g. when remote execution is used.

* Update `SkymeldUsed` to include the combined analysis and execution
phase, as well as optionally the phase at which execution likely starts.
* Update `EstimatedCoresDataProvider` to estimate cores when Skymeld is
used.
* Update `CriticalPathNotDominantSuggestionProvider` when Skymeld is
used, if the information when execution started is available.

Contributes to #97.

Signed-off-by: Sara Adams <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@saraadams saraadams

Labels
type/feat Suggests new features.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@saraadams @lrgoldstein