Difficulties in understanding the meaning of some metrics

[pietrarenner]

Hey everyone!

I have recently started creating graphs using the metrics from Prometheus and OpenTelemetry and I have encountered some difficulties to understand the meaning and the differences between some metrics related to CPU and memory usage. I will list below a few questions I have on this topic. I would be really grateful for anyone who can help me with this questions! :)

1. What is the difference between usage and utilization? I have read the meaning of both concepts in the OpenTelemetry's documentation, but I couldn't really understand the difference.
2. If I want to monitor how much of my CPU has been used (in percentage), should I use metrics like system_cpu_usage and process_cpu_usage or is it better for me to use process_runtime_jvm_cpu_utilization and process_runtime_jvm_system_cpu_utilization?
3. What is the difference between metrics that start with process_runtime_jvm_ and metrics that start only with jvm_? I have read in OpenTelemetry's documentation that any metrics that start with process.runtime. is specific to a certain runtime environment. In that case, I assume that the metrics that start with process_runtime_jvm_ are specific to the JVM environment. I just can't understand in what they differ from the metrics that start with jvm_, shouldn't those metrics be specific to the JVM environment as well?
4. To measure how much memory has been used (in percentage), what metrics should I consider using? The possible metrics are also the ones I am refering to in question 3 and they are: jvm_memory_used, jvm_memory_max, process_runtime_jvm_memory_usage, process_runtime_jvm_memory_limit and process_runtime_jvm_memory_commited.

[trask]

hi @pietrarenner!

1. What is the difference between usage and utilization? I have read the meaning of both concepts in the OpenTelemetry's documentation, but I couldn't really understand the difference.

just checking that this is the doc you read? https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/metrics.md#instrument-naming

2. If I want to monitor how much of my CPU has been used (in percentage), should I use metrics like system_cpu_usage and process_cpu_usage or is it better for me to use process_runtime_jvm_cpu_utilization and process_runtime_jvm_system_cpu_utilization?

if you are using the OpenTelemetry Collector, I'd probably recommend using those metrics as your primary metrics since those can be collected across all services, not only Java services

3. What is the difference between metrics that start with process_runtime_jvm_ and metrics that start only with jvm_? I have read in OpenTelemetry's documentation that any metrics that start with process.runtime. is specific to a certain runtime environment. In that case, I assume that the metrics that start with process_runtime_jvm_ are specific to the JVM environment. I just can't understand in what they differ from the metrics that start with jvm_, shouldn't those metrics be specific to the JVM environment as well?

we recently renamed process.runtime.jvm.* to jvm.*, here are the latest: https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/metrics.md#instrument-naming

4. To measure how much memory has been used (in percentage), what metrics should I consider using? The possible metrics are also the ones I am refering to in question 3 and they are: jvm_memory_used, jvm_memory_max, process_runtime_jvm_memory_usage, process_runtime_jvm_memory_limit and process_runtime_jvm_memory_commited.

if you are using the OpenTelemetry Collector, I'd lean on those metrics first, for the same reason as above

jvm.memory.used / jvm.memory.limit (summed across the dimensions) should give you the percentage of memory used (out of the limit assigned to the JVM, as opposed to out of the limit assigned to the container)

[pietrarenner]

thank you for answering me, @trask!

just checking that this is the doc you read? https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/metrics.md#instrument-naming

yes, that was the doc I read!

jvm.memory.used / jvm.memory.limit (summed across the dimensions) should give you the percentage of memory used (out of the limit assigned to the JVM, as opposed to out of the limit assigned to the container)

I just have one more question about this answer. Do you know if jvm.memory.limit is the same as jvm.memory.max? I got a bit confused, because I do not have any metric named jvm.memory.limit, just one named jvm.memory.max

[trask]

I just have one more question about this answer. Do you know if jvm.memory.limit is the same as jvm.memory.max? I got a bit confused, because I do not have any metric named jvm.memory.limit, just one named jvm.memory.max

yes, .max was renamed to .limit recently in the semantic conventions, if the Java instrumentation isn't emitting the latest name (.limit) can you open an issue in https://github.com/open-telemetry/opentelemetry-java-instrumentation?