-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add cpu/cpuset cgroup and procfs data gathering #11763
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm after CI 👍
we should probably add docs to https://github.com/apache/druid/blob/master/docs/configuration/index.md#enabling-metrics and maybe to https://github.com/apache/druid/blob/master/docs/operations/metrics.md though the latter does not have the other cgroups metrics. im ok if docs are added in a follow-up PR
core/src/main/java/org/apache/druid/java/util/metrics/CgroupCpuMonitor.java
Outdated
Show resolved
Hide resolved
@arunramani should be good to go once we eliminate the possibility of division by zero error. |
It should just return |
Description
Add different ways to collect CPU availability information. This PR gets information from cpu/cpuset cgroups and from /proc/cpuinfo. Also adds 2 more cgroup monitors to emit this info.
The motivation here is to improve the way we figure out available processors since Java's
availableProcessors
implementation doesn't account for fractional quotas, effective CPU sets and shares. This becomes especially important in containerized setups like Kubernetes. Eventually we will extendRuntimeInfo
to be smarter and when we move to Java 11, we can swap to theMetrics
class.Key changed/added classes in this PR
Cpu
andCpuSet
tojava.util.metrics.cgroups
ProcFsUtil
for procfs infoCgroupCpuMonitor
andCgroupCpuSetMonitor
This PR has: