Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(spanner): fix negative values for max_in_use_sessions metrics #10449 #10508

Merged
merged 5 commits into from
Jul 16, 2024
+7 −2
Merged

fix(spanner): fix negative values for max_in_use_sessions metrics #10449 #10508

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
4726a59
fix(spanner): add debug log to print full stack trace when negative v…
rahul2393 Jul 4, 2024
67d9383
skip decrementing num_in_use metric count when session is destroyed f…
rahul2393 Jul 11, 2024
8e32137
Merge branch 'main' into fix_negative_metrics
rahul2393 Jul 15, 2024
c37b045
add back lock
rahul2393 Jul 16, 2024
91685c9
Merge branch 'main' into fix_negative_metrics
rahul2393 Jul 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions spanner/session.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1191,6 +1191,11 @@ func (p *sessionPool) incNumInUseLocked(ctx context.Context) {

func (p *sessionPool) decNumInUseLocked(ctx context.Context) {
p.numInUse--
if int64(p.numInUse) < 0 {
// print whole call stack trace
logf(p.sc.logger, "Number of sessions in use is negative, resetting it to currSessionsCheckedOutLocked. Stack trace: %s", string(debug.Stack()))
p.numInUse = p.currSessionsCheckedOutLocked()
}
p.recordStat(ctx, SessionsCount, int64(p.numInUse), tagNumInUseSessions)
p.recordStat(ctx, ReleasedSessionsCount, 1)
if p.otConfig != nil {
Expand Down Expand Up @@ -1459,12 +1464,12 @@ func (hc *healthChecker) healthCheck(s *session) {
defer hc.markDone(s)
if !s.pool.isValid() {
// Session pool is closed, perform a garbage collection.
s.destroy(false, true)
s.destroy(false, false)
return
}
if err := s.ping(); isSessionNotFoundError(err) {
// Ping failed, destroy the session.
s.destroy(false, true)
s.destroy(false, false)
}
}

Expand Down
Loading