Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add debug storage export endpoint #752

Merged
merged 1 commit into from
Jan 21, 2022
Merged

Conversation

kolesnikovae
Copy link
Collaborator

@kolesnikovae kolesnikovae commented Jan 21, 2022

The PR is aimed at improving debuggability. In order to solve issues like #715 more efficiently we need an option to export data from the storage in a user-friendly way. We extend our /debug endpoints list with /debug/storage/export/{db} that exports values from the storage via HTTP.

Example:

curl -G -o /tmp/segment --data-urlencode "k=s:pyroscope.server.cpu{}" http://localhost:4040/debug/storage/export/segments

The request will write s:pyroscope.server.cpu{} segment raw bytes to /tmp/segment.

NOTE: We have set 15 seconds HTTP server timeout, and it is not configurable. I'm wondering if this should be lifted - say, a minute, or two.

@kolesnikovae kolesnikovae added enhancement New feature or request storage Low level storage matters labels Jan 21, 2022
@kolesnikovae kolesnikovae self-assigned this Jan 21, 2022
@github-actions
Copy link
Contributor

size-limit report 📦

Path Size Loading time (3g) Running time (snapdragon) Total time
webapp/public/assets/app.js 350.25 KB (0%) 7.1 s (0%) 1.1 s (+3.71% 🔺) 8.1 s

@pyroscopebot
Copy link
Collaborator

Screenshots

Throughput Throughput
Disk Usage Disk Usage
Memory Memory
Upload Errors (Total) Upload Errors (Total)
Successful Uploads (Total) Successful Uploads (Total)
CPU Utilization CPU Utilization

Result

main pr diff threshold
throughput 183.62 169.68 🟥 -13.94 (-7.89%) 5%
total items processed 54991.00 50741.00 🟥 -4250.00 (-8.04%) 5%
Details
Name Description Query for main Query for pr
throughput rate(pyroscope_http_request_duration_seconds_count{handler="/ingest", instance="pyroscope-main:4040"}[5m]) rate(pyroscope_http_request_duration_seconds_count{handler="/ingest", instance="pyroscope:4040"}[5m])
total items processed pyroscope_http_request_duration_seconds_count{handler="/ingest", instance="pyroscope-main:4040"} pyroscope_http_request_duration_seconds_count{handler="/ingest", instance="pyroscope:4040"}

Generated by 🚫 dangerJS against 126b70c

@codecov
Copy link

codecov bot commented Jan 21, 2022

Codecov Report

Merging #752 (eb546a7) into main (97ca0f4) will decrease coverage by 0.62%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #752      +/-   ##
==========================================
- Coverage   76.09%   75.48%   -0.61%     
==========================================
  Files          60       61       +1     
  Lines        2070     2112      +42     
  Branches      431      444      +13     
==========================================
+ Hits         1575     1594      +19     
- Misses        465      488      +23     
  Partials       30       30              
Impacted Files Coverage Δ
...mponents/FlameGraph/FlameGraphComponent/Header.tsx 67.45% <0.00%> (-23.03%) ⬇️
...components/FlameGraph/FlameGraphComponent/color.ts 78.13% <0.00%> (-16.42%) ⬇️
...lameGraph/FlameGraphComponent/Flamegraph_render.ts 96.86% <0.00%> (-0.09%) ⬇️
...nts/FlameGraph/FlameGraphComponent/colorPalette.ts 100.00% <0.00%> (ø)
...nents/FlameGraph/FlameGraphComponent/Flamegraph.ts 93.75% <0.00%> (+0.04%) ⬆️
...ents/FlameGraph/FlameGraphComponent/DiffLegend.tsx 89.48% <0.00%> (+0.29%) ⬆️
...omponents/FlameGraph/FlameGraphComponent/index.tsx 85.00% <0.00%> (+0.96%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 97ca0f4...eb546a7. Read the comment docs.

@petethepig petethepig merged commit 5040fb3 into main Jan 21, 2022
@petethepig petethepig deleted the feat/debug_storage_export branch January 21, 2022 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request storage Low level storage matters
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants