Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

roachprod: commands for pprof profiling #62309

Closed
erikgrinaker opened this issue Mar 20, 2021 · 0 comments · Fixed by #62796
Closed

roachprod: commands for pprof profiling #62309

erikgrinaker opened this issue Mar 20, 2021 · 0 comments · Fixed by #62796
Assignees
@stevendanna
Labels
A-debug-automation Issues or feature requests for debug automation A-roachprod C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)

Comments

@erikgrinaker
Copy link
Contributor

Clusters set up with roachprod have pprof enabled and available at the admin URL given by roachprod adminurl under the path /debug/pprof. It would be useful with a few roachprod commands that automatically collected pprof profiles, e.g.:

  • roachprod pprofurl: outputs the pprof URL.
  • roachprod pprof-profile: automatically takes a pprof CPU profile, dumps it to a local timestamped file, generates e.g. a PDF file, and opens the pprof web UI.
  • roachprod pprof-heap: similar to pprof-profile, but for a heap profile.
@erikgrinaker erikgrinaker added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-roachprod A-debug-automation Issues or feature requests for debug automation labels Mar 20, 2021
@erikgrinaker erikgrinaker mentioned this issue Mar 21, 2021
Closed
15 tasks
@stevendanna stevendanna self-assigned this Mar 30, 2021
@stevendanna stevendanna mentioned this issue Mar 30, 2021
Merged
stevendanna added a commit to stevendanna/cockroach that referenced this issue Mar 30, 2021
@stevendanna
roachprod: add pprof command
ff99b82 
The new pprof command allows to user to collect CPU and heap profiles
from nodes in a roachprod cluster using pprof's HTTP endpoints.

Profiles are collected in parallel and stored in the current working
directory of the machine running pprof. The command can also
optionally open the profiles using `go tool pprof`.

Some usage examples:

    # Capture CPU profile for all nodes in the cluster
    roachprod pprof CLUSTERNAME
    # Capture CPU profile for the first node in the cluster for 60 seconds
    roachprod pprof CLUSTERNAME:1 --duration 60s
    # Capture a Heap profile for the first node in the cluster
    roachprod pprof CLUSTERNAME:1 --heap
    # Same as above
    roachprod pprof-heap CLUSTERNAME:1

Fixes cockroachdb#62309

Release note: None
stevendanna added a commit to stevendanna/cockroach that referenced this issue Mar 30, 2021
@stevendanna
roachprod: add pprof command
0a5f420 
The new pprof command allows to user to collect CPU and heap profiles
from nodes in a roachprod cluster using pprof's HTTP endpoints.

Profiles are collected in parallel and stored in the current working
directory of the machine running pprof. The command can also
optionally open the profiles using `go tool pprof`.

Some usage examples:

    # Capture CPU profile for all nodes in the cluster
    roachprod pprof CLUSTERNAME
    # Capture CPU profile for the first node in the cluster for 60 seconds
    roachprod pprof CLUSTERNAME:1 --duration 60s
    # Capture a Heap profile for the first node in the cluster
    roachprod pprof CLUSTERNAME:1 --heap
    # Same as above
    roachprod pprof-heap CLUSTERNAME:1

Fixes cockroachdb#62309

Release note: None
stevendanna added a commit to stevendanna/cockroach that referenced this issue Mar 30, 2021
@stevendanna
roachprod: add pprof command
d348ad1 
The new pprof command allows to user to collect CPU and heap profiles
from nodes in a roachprod cluster using pprof's HTTP endpoints.

Profiles are collected in parallel and stored in the current working
directory of the machine running pprof. The command can also
optionally open the profiles using `go tool pprof`.

Some usage examples:

    # Capture CPU profile for all nodes in the cluster
    roachprod pprof CLUSTERNAME
    # Capture CPU profile for the first node in the cluster for 60 seconds
    roachprod pprof CLUSTERNAME:1 --duration 60s
    # Capture a Heap profile for the first node in the cluster
    roachprod pprof CLUSTERNAME:1 --heap
    # Same as above
    roachprod pprof-heap CLUSTERNAME:1

Fixes cockroachdb#62309

Release note: None
craig bot pushed a commit that referenced this issue Mar 31, 2021
@yuzefovich @stevendanna
Merge #62640 #62796
00e8d9c 
62640: colexecutils: optimize the deselector a bit for zero batch r=yuzefovich a=yuzefovich

Zero length batch is special in the vectorized engine. Once we receive
it from the input, in the vast majority of cases we don't need to do any
processing in the operator. Previously, the deselector would still do
some stuff even for zero batch, and now it'll short-circuit as most
operators do.

Release note: None

62796: roachprod: add pprof command r=erikgrinaker a=stevendanna

The new pprof command allows to user to collect CPU and heap profiles
from nodes in a roachprod cluster using pprof's HTTP endpoints.

Profiles are collected in parallel and stored in the current working
directory of the machine running pprof. The command can also
optionally open the profiles using `go tool pprof`.

Some usage examples:

    # Capture CPU profile for all nodes in the cluster
    roachprod pprof CLUSTERNAME
    # Capture CPU profile for the first node in the cluster for 60 seconds
    roachprod pprof CLUSTERNAME:1 --duration 60s
    # Capture a Heap profile for the first node in the cluster
    roachprod pprof CLUSTERNAME:1 --heap
    # Same as above
    roachprod pprof-heap CLUSTERNAME:1

Fixes #62309

Note that I've skipped the `pprofurl` command in the original issue. I noticed that the `adminurl` 
command has an option to append any string you'd like to the URL, which is probably most of what one
needs from that command in most cases. But, I can add it if others feel differently.

Release note: None

Co-authored-by: Yahor Yuzefovich <[email protected]>
Co-authored-by: Steven Danna <[email protected]>
@craig craig bot closed this as completed in 1d2b57e Mar 31, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@stevendanna stevendanna

Labels
A-debug-automation Issues or feature requests for debug automation A-roachprod C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

roachprod: add pprof command stevendanna/cockroach
2 participants
@erikgrinaker @stevendanna