Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] Produce and upload bazel execution logs #20318

Draft
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

pamaury
Copy link
Contributor

@pamaury pamaury commented Nov 10, 2023

Execution logs can be used to debug caching and understand better what bazel is doing. This commit introduces a general mecanism to upload those logs. Only convert the SW build and test step for now.

@pamaury pamaury requested a review from a-will November 10, 2023 18:38
@pamaury pamaury force-pushed the bazel_cache_investigate branch from 0e15c51 to 9701dd8 Compare November 10, 2023 18:51
@a-will
Copy link
Contributor

a-will commented Nov 10, 2023

This looks interesting! However, it appears bazel struggles to handle the data within whatever memory size restrictions we have set up (not sure if it's physical or JVM-limited). I wonder why.

(19:28:52) INFO: Build completed successfully, 16133 total actions
(19:30:08) WARNING: Execution log might not have been populated. Raw execution log is at /tmp/exec4159515575069434779.log
FATAL: bazel ran out of memory and crashed.

@pamaury pamaury force-pushed the bazel_cache_investigate branch from 9701dd8 to 7333ba7 Compare November 11, 2023 02:31
@pamaury
Copy link
Contributor Author

pamaury commented Nov 11, 2023

I think this is a know problem and, in a typical bazel fashion, there is a flag for it ! that may help in certain circumstances...

@pamaury
Copy link
Contributor Author

pamaury commented Nov 12, 2023

The execution log is 1.8GB large :-o

Execution logs can be used to debug caching and understand better
what bazel is doing. This commit introduces a general mecanism
to upload those logs. Only convert the SW build and test step for
now.

Signed-off-by: Amaury Pouly <[email protected]>
@pamaury pamaury force-pushed the bazel_cache_investigate branch 5 times, most recently from cef9594 to 400cd90 Compare January 16, 2024 11:18
@pamaury pamaury force-pushed the bazel_cache_investigate branch from 400cd90 to f2a4caa Compare January 16, 2024 11:39
@pamaury pamaury force-pushed the bazel_cache_investigate branch from 9d00bdd to f1023ac Compare January 16, 2024 16:57
@pamaury pamaury force-pushed the bazel_cache_investigate branch from f1023ac to ba29580 Compare January 16, 2024 17:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants