Skip to content
This repository has been archived by the owner on Jun 23, 2022. It is now read-only.

feat: update the way to get heap profile #433

Merged
merged 16 commits into from
Apr 28, 2020
Merged

feat: update the way to get heap profile #433

Changes from 7 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
94a8072
update heap profile test
hycdong Apr 1, 2020
2f246cc
update code
hycdong Apr 3, 2020
0b885b5
Merge branch 'master' into heap_profile
levy5307 Apr 8, 2020
9eaa36f
Merge branch 'master' into heap_profile
hycdong Apr 8, 2020
3a4e8f0
Merge branch 'master' into heap_profile
Apr 10, 2020
02b09b9
Merge branch 'pegasus' into heap_profile
hycdong Apr 17, 2020
1c67745
update default second
hycdong Apr 17, 2020
8540039
fix by review
hycdong Apr 23, 2020
9401a77
add concurrent check for some pprof action
hycdong Apr 24, 2020
dcbd450
Merge branch 'master' into heap_profile
hycdong Apr 24, 2020
03c6a0c
Merge branch 'master' into heap_profile
hycdong Apr 24, 2020
347be03
Merge branch 'master' into heap_profile
acelyc111 Apr 24, 2020
0440c90
update to CAS operation
hycdong Apr 26, 2020
f982880
Merge branch 'master' into heap_profile
hycdong Apr 26, 2020
7ffc9af
Merge branch 'master' into heap_profile
hycdong Apr 27, 2020
d479f26
Merge branch 'master' into heap_profile
acelyc111 Apr 27, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 23 additions & 15 deletions src/dist/http/pprof_http_service.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
#include <dsn/utility/defer.h>
#include <dsn/utility/timer.h>
#include <dsn/utility/string_splitter.h>
#include <gperftools/heap-profiler.h>
#include <gperftools/malloc_extension.h>
#include <gperftools/profiler.h>

Expand Down Expand Up @@ -320,25 +321,32 @@ void pprof_http_service::symbol_handler(const http_request &req, http_response &
// == ip:port/pprof/heap == //
// //

static constexpr const char *TCMALLOC_SAMPLE_PARAMETER = "TCMALLOC_SAMPLE_PARAMETER";

static bool is_heap_profile_enabled() { return ::getenv(TCMALLOC_SAMPLE_PARAMETER) != nullptr; }

static bool get_heap_profile(std::string &result)
void pprof_http_service::heap_handler(const http_request &req, http_response &resp)
{
if (!is_heap_profile_enabled()) {
result = "no TCMALLOC_SAMPLE_PARAMETER in env";
return false;
const std::string SECOND = "seconds";
neverchanje marked this conversation as resolved.
Show resolved Hide resolved
const int defaultSecond = 10;
hycdong marked this conversation as resolved.
Show resolved Hide resolved

// get seconds from query params, default value is `defaultSecond`
int seconds = defaultSecond;
const auto iter = req.query_args.find(SECOND);
if (iter != req.query_args.end()) {
const auto seconds_str = iter->second;
if (!seconds_str.empty()) {
seconds = std::atoi(seconds_str.c_str());
hycdong marked this conversation as resolved.
Show resolved Hide resolved
}
}
MallocExtension::instance()->GetHeapSample(&result);
return true;
}

void pprof_http_service::heap_handler(const http_request &req, http_response &resp)
{
resp.status_code = http_status_code::ok;
std::stringstream profile_name_prefix;
profile_name_prefix << "heap_profile." << getpid() << "." << dsn_now_ns();

HeapProfilerStart(profile_name_prefix.str().c_str());
sleep(seconds);
const char *profile = GetHeapProfile();
HeapProfilerStop();

get_heap_profile(resp.body);
resp.status_code = http_status_code::ok;
resp.body = profile;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Has the problem that response too large and http server can not deal with been solved?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have read source code, and find that GetHeapProfile will return buffers at most 1MB.

// We use buffers of this size in DoGetHeapProfile.
static const int kProfileBufferSize = 1 << 20;

During my test, I have observed that some middle profile files whose size will not exceed 1M, it proves the file size limit.
Besides, I have read google pprof source code roughly, find that pprof toolset will collect several heap profile files, and combine them into one file, then serialize, compress the file. I suppose that pprof toolset did something to get whole profile file.

delete profile;
}

// //
Expand Down