-
Notifications
You must be signed in to change notification settings - Fork 219
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
buck2_client: download traces via buck2.log_url
config key
#770
Draft
thoughtpolice
wants to merge
1
commit into
facebook:main
Choose a base branch
from
thoughtpolice:aseipp/push-xnmzuqpwomuk
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
buck2_client: download traces via buck2.log_url
config key
#770
thoughtpolice
wants to merge
1
commit into
facebook:main
from
thoughtpolice:aseipp/push-xnmzuqpwomuk
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
Sep 5, 2024
thoughtpolice
force-pushed
the
aseipp/push-xnmzuqpwomuk
branch
from
September 5, 2024 00:32
7d26fab
to
b112eb3
Compare
This can't do things like pass authentication credentials just yet, but I'd like to know if this is the right shape. As it is, it's still incredibly useful, so I'd like to know if it looks like roughly the right thing before doing much more. |
thoughtpolice
force-pushed
the
aseipp/push-xnmzuqpwomuk
branch
from
September 5, 2024 01:53
b112eb3
to
bc3001f
Compare
thoughtpolice
changed the title
buck2_client: download traces via
buck2_client: download traces via Sep 5, 2024
buck2.log_download_url
config keybuck2.log_url
config key
I'd like to make some small API changes to the GET URI so I'm holding this. But I'd still really like something like this to go in :) |
thoughtpolice
force-pushed
the
aseipp/push-xnmzuqpwomuk
branch
4 times, most recently
from
September 8, 2024 22:26
ba03f55
to
505c4c2
Compare
thoughtpolice
force-pushed
the
aseipp/push-xnmzuqpwomuk
branch
2 times, most recently
from
September 20, 2024 22:15
9206ebd
to
7e86204
Compare
thoughtpolice
force-pushed
the
aseipp/push-xnmzuqpwomuk
branch
from
September 29, 2024 01:28
7e86204
to
58a82a3
Compare
thoughtpolice
force-pushed
the
aseipp/push-xnmzuqpwomuk
branch
2 times, most recently
from
October 16, 2024 20:35
0949481
to
947d903
Compare
When doing a command like `buck2 log replay --trace-id ...` it helps to be able to download previously created logs from users, automated CI, etc. This functionality exists within Meta, as logs are available for download through the "Manifest" system, but it isn't usable in OSS, despite being useful for diagnostics and getting help. The only missing thing to really get things working is a download mechanism for log files, and a way to know where they should come from. With this patch, if a user configures the `buck2.log_url` key, which is expected to look something like: [buck2] log_url = https://example.com Then, upon executing a `log` command with a `--trace-id` flag, this server will be queried with a: GET /v1/get/{uuid} request, which is expected to return the raw zst-encoded protobuf file. So, buck2 will do that and download the trace, and then use it like normal. The request is done with `curl`, though in principle it could be done within Buck itself. This shares as much code as possible with the existing infrastructure and tries to only insert a small key set of `#[cfg(fbcode_build)]` directives. Notably, the path Meta uses for their "Manifold" client is still fully available in the OSS version; `fbcode_build` is only used to gate what the default choice is. How the server gets access to these files and how they are uploaded is another question. But for now, this can be done several ways outside of buck2 core, so this is good enough. GitHub Issue: facebook#441 Signed-off-by: Austin Seipp <[email protected]>
thoughtpolice
force-pushed
the
aseipp/push-xnmzuqpwomuk
branch
from
October 20, 2024 20:35
947d903
to
bee85c8
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When doing a command like
buck2 log replay --trace-id ...
it helps to be able to download previously created logs from users, automated CI, etc. This functionality exists within Meta, as logs are available for download through the "Manifest" system, but it isn't usable in OSS, despite being useful for diagnostics and getting help.The only missing thing to really get things working is a download mechanism for log files, and a way to know where they should come from.
With this patch, if a user configures the
buck2.log_url
key, which is expected to look something like:Then, upon executing a
log
command with a--trace-id
flag, this server will be queried with a:request, which is expected to return the raw zst-encoded protobuf file. So, buck2 will do that and download the trace, and then use it like normal.
The request is done with
curl
, though in principle it could be done within Buck itself.This shares as much code as possible with the existing infrastructure and tries to only insert a small key set of
#[cfg(fbcode_build)]
directives. Notably, the path Meta uses for their "Manifold" client is still fully available in the OSS version;fbcode_build
is only used to gate what the default choice is.How the server gets access to these files and how they are uploaded is another question. But for now, this can be done several ways outside of buck2 core, so this is good enough.
GitHub Issue: #441