HBASE-27948 Report memstore on-heap and off-heap size as jmx metrics in sub=Memory bean #5293
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
💔 -1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
Apache9
reviewed
Jun 16, 2023
| @@ -118,6 +130,13 @@ public interface MetricsHeapMemoryManagerSource extends BaseSource { | |||
| String UNBLOCKED_FLUSH_GAUGE_DESC = "Gauge for the unblocked flush count before tuning"; | |||
| String MEMSTORE_SIZE_GAUGE_NAME = "memStoreSize"; | |||
| String MEMSTORE_SIZE_GAUGE_DESC = "Global MemStore used in bytes by the RegionServer"; | |||
| String MEMSTORE_ONHEAP_SIZE_GAUGE_NAME = "memStoreOnHeapSize"; | |||
There was a problem hiding this comment.
Use OnHeap or just Heap? I'm not an English expert, just asking...
There was a problem hiding this comment.
Both Heap and OnHeap are used to referring to on-heap size in our code, so I think both are okay, and choose OnHeap here to differentiate it with OffHeap.
| tunerContext.setCurMemStoreUsed((float) globalMemstoreHeapSize / maxHeapSize); | ||
| metricsHeapMemoryManager.setCurMemStoreSizeGauge(globalMemstoreHeapSize); |
There was a problem hiding this comment.
So in the old time we just use heap size as memstore size? This should be bug?
There was a problem hiding this comment.
yes, this is what i was wondering.
FYI @bbeaudreault as he discovered this
There was a problem hiding this comment.
Ah interesting.
So I wasn't talking about these metrics at all. These metrics show up in JMX under sub=Memory (which for us is all 0's for some reason, and we don't use them).
The metrics I was referring to are in sub=Server and also per-region and per-table metrics. These also have a memStoreSize field, and it is derived from the DataSize rather than HeapSize. These are calculated in a few places, you have to sort of dig in based on usages of MetricsRegionServerSource.MEMSTORE_SIZE
There was a problem hiding this comment.
Sorry for the confusion, I didn't realize these metrics existed. Maybe they are better? I need to read more.
There was a problem hiding this comment.
i was not aware either, realized these metrics exist and this is likely bug only after Jing created this PR, i also need to do some digging here.
Thanks for pointing out sub=Server @bbeaudreault
FYI @jinggou
There was a problem hiding this comment.
@bbeaudreault Sorry I didn't realize that you are referring to sub=Server. This change is made for sub=Memory, so I create a new pr (#5308) for HBASE-27892.
For the code change here, I think it might be a bug because memStoreSize should refer to memstore data size instead of memstore heap size, and also noticed that its value is 0 under sub=Memory (while not 0 under sub=Server). I've created a new jira HBASE-27948 to figure out this issue. @Apache9 @virajjasani
jinggou
changed the title
@jinggou what about the new metrics that you introduced with this PR? they are also being reported as 0 under Memory section? |
Yes. |
this needs work, maybe in separate Jira |
virajjasani
approved these changes
Jun 28, 2023
virajjasani
pushed a commit
that referenced
this pull request
Jun 29, 2023
…in sub=Memory bean (#5293) Signed-off-by: Viraj Jasani <[email protected]>
virajjasani
pushed a commit
that referenced
this pull request
Jun 29, 2023
…in sub=Memory bean (#5293) Signed-off-by: Viraj Jasani <[email protected]>
virajjasani
pushed a commit
that referenced
this pull request
Jun 29, 2023
…in sub=Memory bean (#5293) Signed-off-by: Viraj Jasani <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Jira: HBASE-27948