executor: add some memory tables to observe memory usage #38452
Conversation
|
[REVIEW NOTIFICATION] This pull request has been approved by:
To complete the pull request process, please ask the reviewers in the list to review by filling The full list of commands accepted by this bot can be found here. Reviewer can indicate their review by submitting an approval review. |
ti-chi-bot
added
release-note-none
ti-chi-bot
removed
the
do-not-merge/work-in-progress
ti-chi-bot
added
the
needs-rebase
ti-chi-bot
removed
the
needs-rebase
| sqlInfo.SetString(fmt.Sprintf("%.256v", sqlInfo.GetString()), mysql.DefaultCollationName) // Truncated | ||
|
|
||
| m.infos = append(m.infos, op) | ||
| if len(m.infos) > 50 { |
There was a problem hiding this comment.
We do not have an eviction strategy for now.
Maybe we can define m.infos = make([]*memoryOps, 50, 50) directly. Every time we write the new record in m.infos[offset]. When querying, output [m.infos:length) + [0,m.infos].
util/memory/tracker.go
Outdated
| QueryForceDisk = atomicutil.NewInt64(0) | ||
| TriggerMemoryLimitGC = atomicutil.NewBool(false) | ||
| MemoryLimitGCLast = atomicutil.NewTime(time.Time{}) | ||
| MemoryLimitGCToTal = atomicutil.NewInt64(0) |
There was a problem hiding this comment.
| MemoryLimitGCToTal = atomicutil.NewInt64(0) | |
| MemoryLimitGCTotal = atomicutil.NewInt64(0) | |
| Write |
ti-chi-bot
added
size/XL
executor/infoschema_reader.go
Outdated
| @@ -2398,6 +2409,67 @@ func (e *memtableRetriever) setDataForClusterTrxSummary(ctx sessionctx.Context) | |||
| return nil | |||
| } | |||
|
|
|||
| func (e *memtableRetriever) setDataForMemoryUsage(ctx sessionctx.Context) error { | |||
| r := &runtime.MemStats{} | |||
| runtime.ReadMemStats(r) | |||
There was a problem hiding this comment.
Although we have to use it, I'd like to point out that ReadMemStats is not a free-lunch, it calls "stop the world" ...
| for i := m.offsets; i < len(m.infos); i++ { | ||
| if m.infos[i].killTime.Equal(zeroTime) { | ||
| continue | ||
| } | ||
| getRowFromInfo(m.infos[i]) | ||
| } | ||
| for i := 0; i < m.offsets; i++ { | ||
| getRowFromInfo(m.infos[i]) | ||
| } |
There was a problem hiding this comment.
Iterate the queue backward to the first element?
for i :=0; i<50; i++ {
pos := [m.offsets + 50 - i] % 50
info := m.infos[pos]
if info.killTime.Equal(zeroTime) {
break
}
getRowFromInfo(info)
...
}
| @@ -58,6 +62,8 @@ var memTableToClusterTables = map[string]string{ | |||
| TableTiDBTrx: ClusterTableTiDBTrx, | |||
| TableDeadlocks: ClusterTableDeadlocks, | |||
| TableTrxSummary: ClusterTableTrxSummary, | |||
| TableMemoryUsage: ClusterTableMemoryUsage, | |||
There was a problem hiding this comment.
Is there a test to cover the basic
"select * from MEMORY_USAGE" or from TableMemoryUsageOpsHistory?
ClusterTableMemoryXXX might be harder, but I think we can cover the table with non-cluster-perfix
ti-chi-bot
added
status/LGT2
|
/merge |
|
This pull request has been accepted and is ready to merge. Commit hash: 7f50d50
|
ti-chi-bot
added
the
status/can-merge
TiDB MergeCI notify✅ Well Done! New fixed [2] after this pr merged.
|
What problem does this PR solve?
Issue Number: ref #37816
Problem Summary:
What is changed and how it works?
Add two memory usage information_schema.MEMORY_USAGE and information_schema.MEMORY_USAGE_OPS_HISTORY to observe memory usage.
Check List
Tests
Side effects
Documentation
Release note
Please refer to Release Notes Language Style Guide to write a quality release note.