-
Notifications
You must be signed in to change notification settings - Fork 989
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
zgc pause time includes concurrent time #2372
Comments
I think isConcurrentPhase is wrong for zgc
https://github.com/Netflix/spectator/blob/3d428b3c69c80f7067772e27ce0032cf11d8ec9c/spectator-ext-gc/src/main/java/com/netflix/spectator/gc/GcLogger.java |
Thank you for the report. Things do seem probably out of sorts. I'd like to check on the ZGC devs mailing list what the correct way is to separate out concurrent and pause time from gc notifications. |
I also just came across JDK-8255289. The comments seem to indicate it's no longer possible in the latest implementations to distinguish pause time from concurrent time using the gc notifications for ZGC. This is unfortunate, especially since there doesn't seem to be an alternative available currently either. |
Make a suggestion,For ZGC, the allocation stall seems to indicate that some thread or threads are blocked trying to allocate. Even though it is not a true STW pause, counting it as such seems to be less confusing. |
I think that comment from the Spectator code is based on the behavior before the change mentioned in a comment on JDK-8255289:
If you run an application with gc logging on you can see the pause time and concurrent time separately. The gc notification is including the concurrent time for ZGC cycles, even when the cause
Edit: To be clear, our metric is showing 73 milliseconds of pause time, but you can see from the GC logs that the 3 pauses add up to less than 1 millisecond - the rest is concurrent phases. Given how wrong the pause time metric is for ZGC, I'm strongly considering removing support for ZGC unless we can find a way to get the information. Edit: I should note that I'm running the above code with Java 15.0.1. |
I don't think so for removing support ZGC, JVM pause time is very important for the application's health;
Also I think the allocation stall is very import for a java application when it apply memory. |
I agree. The issue is we cannot get that information.
No because ZGC is also reporting there the total collection time, including pauses and concurrent phases.
The last line is what is reported from the |
It appears that in JDK 17 the |
@macqm Thanks for sharing. That's good to know. I'll have to take another look at our gc metrics with zgc in 17. |
Since Java 17, the GarbageCollectorMXBean for ZGC has been split into `ZGC Pauses` and `ZGC Cycles`, which allows us to properly measure the pause time and the concurrent collection time. Before we only had access to the overall time. Resolves micrometer-metricsgh-2372
The fix will be in the upcoming 1.6.12, 1.7.5, and 1.8.0 releases. It is available in the corresponding snapshots for early testing. You need to be running Java 17 or later for the fix to have any effect, since that is the release from which the information we needed was made available via JMX. |
To be clear, finally GC pause metric for ZGC is not available now? |
It's available as long as you are using a recent supported version of Micrometer and Java 17 or later. It's available before that also, but the metric is not accurate, since it includes concurrent time as well as pause time. |
Yes, I see that. Thanks I found solution for that. |
Dear @krystianzybala , Could u please kindly let me know what's ur solution? We encounter the same issue in JDK11 with ZGC. Thanks, |
formula:rate(jvm_gc_pause_seconds_sum{application="$application", instance="$instance"}[1m])/rate(jvm_gc_pause_seconds_count{application="$application", instance="$instance"}[1m])
question:
The STW time of zgc is less 10ms, but why jvm_gc_pause_time is 822ms? can not understand this phenomenon
The text was updated successfully, but these errors were encountered: