Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DATA RACE in the TestGlobalMemoryControlForAnalyze #40946

Open
Tracked by #41316
hawkingrei opened this issue Feb 1, 2023 · 3 comments · May be fixed by #41548
Open
Tracked by #41316

DATA RACE in the TestGlobalMemoryControlForAnalyze #40946

hawkingrei opened this issue Feb 1, 2023 · 3 comments · May be fixed by #41548
Assignees
@hawkingrei
Labels
affects-6.6 affects-7.0 affects-7.1 affects-7.5 affects-8.1 component/test severity/moderate sig/planner SIG: Planner type/bug The issue is confirmed as a bug.

Comments

@hawkingrei
Copy link
Member

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

2. What did you expect to see? (Required)

3. What did you see instead (Required)

==================
WARNING: DATA RACE
Write at 0x00c020ef9de0 by goroutine 911520:
  github.com/pingcap/tidb/executor.ResetContextOfStmt()
      executor/executor.go:2052 +0x824
  github.com/pingcap/tidb/session.(*session).ExecuteStmt()
      session/session.go:2142 +0x39c
  github.com/pingcap/tidb/testkit.(*TestKit).ExecWithContext()
      testkit/testkit.go:321 +0x96e
  github.com/pingcap/tidb/testkit.(*TestKit).Exec()
      testkit/testkit.go:294 +0x4db
  executor/analyzetest/analyzetest_test.TestGlobalMemoryControlForAnalyze()
      executor/analyzetest/analyze_test.go:3094 +0x49f
  github.com/pingcap/failpoint.(*Failpoint).Enable()
      external/com_github_pingcap_failpoint/failpoint.go:54 +0x44
  github.com/pingcap/failpoint.(*Failpoints).Enable()
      external/com_github_pingcap_failpoint/failpoints.go:105 +0x276
  github.com/pingcap/failpoint.Enable()
      external/com_github_pingcap_failpoint/failpoints.go:222 +0x435
  executor/analyzetest/analyzetest_test.TestGlobalMemoryControlForAnalyze()
      executor/analyzetest/analyze_test.go:3089 +0x436
  github.com/pingcap/failpoint.(*Failpoint).Enable()
      external/com_github_pingcap_failpoint/failpoint.go:54 +0x44
  github.com/pingcap/failpoint.(*Failpoints).Enable()
      external/com_github_pingcap_failpoint/failpoints.go:105 +0x276
  github.com/pingcap/failpoint.Enable()
      external/com_github_pingcap_failpoint/failpoints.go:222 +0x3ef
  executor/analyzetest/analyzetest_test.TestGlobalMemoryControlForAnalyze()
      executor/analyzetest/analyze_test.go:3088 +0x3f0
  github.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelay()
      executor/adapter.go:779 +0x34a
  github.com/pingcap/tidb/executor.(*ExecStmt).Exec()
      executor/adapter.go:574 +0x1299
  github.com/pingcap/tidb/session.runStmt()
      session/session.go:2351 +0x761
  github.com/pingcap/tidb/session.(*session).ExecuteStmt()
      session/session.go:2215 +0xfa5
  github.com/pingcap/tidb/testkit.(*TestKit).ExecWithContext()
      testkit/testkit.go:321 +0x96e
  github.com/pingcap/tidb/testkit.(*TestKit).MustExecWithContext()
      testkit/testkit.go:130 +0xb7
  github.com/pingcap/tidb/testkit.(*TestKit).MustExec()
      testkit/testkit.go:125 +0xf7
  executor/analyzetest/analyzetest_test.TestGlobalMemoryControlForAnalyze()
      executor/analyzetest/analyze_test.go:3073 +0xda
  github.com/pingcap/tidb/sessionctx/variable.glob..func168()
      sessionctx/variable/sysvar.go:882 +0x5e
  github.com/pingcap/tidb/sessionctx/variable.(*SysVar).Validate()
      sessionctx/variable/variable.go:301 +0x12f
  github.com/pingcap/tidb/session.(*session).SetGlobalSysVar()
      session/session.go:1493 +0xe4
  github.com/pingcap/tidb/executor.(*SetExecutor).setSysVariable()
      executor/set.go:149 +0x6cd
  github.com/pingcap/tidb/executor.(*SetExecutor).Next()
      executor/set.go:99 +0xf59
  github.com/pingcap/tidb/executor.Next()
      executor/executor.go:328 +0x5c3
  github.com/pingcap/tidb/executor.(*ExecStmt).next()
      executor/adapter.go:1198 +0x89
  github.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelayExecutor()
      executor/adapter.go:950 +0x6f1
  github.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelay()
      executor/adapter.go:779 +0x34a
  github.com/pingcap/tidb/executor.(*ExecStmt).Exec()
      executor/adapter.go:574 +0x1299
  github.com/pingcap/tidb/session.runStmt()
      session/session.go:2351 +0x761
  github.com/pingcap/tidb/session.(*session).ExecuteStmt()
      session/session.go:2215 +0xfa5
  github.com/pingcap/tidb/testkit.(*TestKit).ExecWithContext()
      testkit/testkit.go:321 +0x96e
  github.com/pingcap/tidb/testkit.(*TestKit).MustExecWithContext()
      testkit/testkit.go:130 +0xb7
  github.com/pingcap/tidb/testkit.(*TestKit).MustExec()
      testkit/testkit.go:125 +0xf7
  executor/analyzetest/analyzetest_test.TestGlobalMemoryControlForAnalyze()
      executor/analyzetest/analyze_test.go:3072 +0xba
  github.com/pingcap/tidb/session.runStmt()
      session/session.go:2351 +0x761
  github.com/pingcap/tidb/session.(*session).ExecuteStmt()
      session/session.go:2215 +0xfa5
  github.com/pingcap/tidb/testkit.(*TestKit).ExecWithContext()
      testkit/testkit.go:321 +0x96e
  github.com/pingcap/tidb/testkit.(*TestKit).MustExecWithContext()
      testkit/testkit.go:130 +0xb7
  github.com/pingcap/tidb/testkit.(*TestKit).MustExec()
      testkit/testkit.go:125 +0xf7
  executor/analyzetest/analyzetest_test.TestGlobalMemoryControlForAnalyze()
      executor/analyzetest/analyze_test.go:3071 +0x9a
  github.com/pingcap/tidb/domain.(*Domain).rebuildSysVarCache()
      domain/sysvar_cache.go:146 +0x844
  github.com/pingcap/tidb/domain.(*Domain).LoadSysVarCacheLoop()
      domain/domain.go:1384 +0xa8
  github.com/pingcap/tidb/session.BootstrapSession()
      session/session.go:3337 +0x693
  github.com/pingcap/tidb/domain.(*Domain).GetSessionCache()
      domain/sysvar_cache.go:62 +0x59
  github.com/pingcap/tidb/session.(*session).loadCommonGlobalVariablesIfNeeded()
      session/session.go:3656 +0x104
  github.com/pingcap/tidb/session.(*session).ExecuteStmt()
      session/session.go:2134 +0x2a5
  github.com/pingcap/tidb/session.(*session).ExecuteInternal()
      session/session.go:1668 +0x471
  github.com/pingcap/tidb/domain.(*Domain).LoadPrivilegeLoop()
      domain/domain.go:1328 +0x130
  github.com/pingcap/tidb/session.BootstrapSession()
      session/session.go:3330 +0x644
  github.com/pingcap/tidb/testkit.bootstrap()
      testkit/mockstore.go:85 +0xac
  github.com/pingcap/tidb/testkit.CreateMockStoreAndDomain()
      testkit/mockstore.go:70 +0xe9
  executor/analyzetest/analyzetest_test.TestGlobalMemoryControlForAnalyze()
      executor/analyzetest/analyze_test.go:3068 +0x55
  testing.tRunner()
      GOROOT/src/testing/testing.go:1446 +0x216
  testing.(*T).Run.func1()
      GOROOT/src/testing/testing.go:1493 +0x47
Previous read at 0x00c020ef9de0 by goroutine 924724:
  github.com/pingcap/tidb/util/servermemorylimit.killSessIfNeeded()
      util/servermemorylimit/servermemorylimit.go:125 +0x2a7
  github.com/pingcap/tidb/util/servermemorylimit.(*Handle).Run()
      util/servermemorylimit/servermemorylimit.go:74 +0x148
  executor/analyzetest/analyzetest_test.TestGlobalMemoryControlForAnalyze.func2()
      executor/analyzetest/analyze_test.go:3079 +0x39
Goroutine 911520 (running) created at:
  testing.(*T).Run()
      GOROOT/src/testing/testing.go:1493 +0x75d
  testing.runTests.func1()
      GOROOT/src/testing/testing.go:1846 +0x99
  testing.tRunner()
      GOROOT/src/testing/testing.go:1446 +0x216
  testing.runTests()
      GOROOT/src/testing/testing.go:1844 +0x7ec
  testing.(*M).Run()
      GOROOT/src/testing/testing.go:1726 +0xa84
  go.uber.org/goleak.VerifyTestMain()
      external/org_uber_go_goleak/testmain.go:53 +0x70
  executor/analyzetest/analyzetest_test.TestMain()
      executor/analyzetest/main_test.go:29 +0x318
  main.main()
      bazel-out/k8-fastbuild/bin/executor/analyzetest/analyzetest_test_/testmain.go:212 +0x5e8
Goroutine 924724 (running) created at:
  executor/analyzetest/analyzetest_test.TestGlobalMemoryControlForAnalyze()
      executor/analyzetest/analyze_test.go:3079 +0x331
  github.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelay()
      executor/adapter.go:779 +0x34a
  github.com/pingcap/tidb/executor.(*ExecStmt).Exec()
      executor/adapter.go:574 +0x1299
  github.com/pingcap/tidb/session.runStmt()
      session/session.go:2351 +0x761
  github.com/pingcap/tidb/session.(*session).ExecuteStmt()
      session/session.go:2215 +0xfa5
  github.com/pingcap/tidb/testkit.(*TestKit).ExecWithContext()
      testkit/testkit.go:321 +0x96e
  github.com/pingcap/tidb/testkit.(*TestKit).MustExecWithContext()
      testkit/testkit.go:130 +0xb7
  github.com/pingcap/tidb/testkit.(*TestKit).MustExec()
      testkit/testkit.go:125 +0xf7
  executor/analyzetest/analyzetest_test.TestGlobalMemoryControlForAnalyze()
      executor/analyzetest/analyze_test.go:3073 +0xda
  github.com/pingcap/tidb/sessionctx/variable.glob..func168()
      sessionctx/variable/sysvar.go:882 +0x5e
  github.com/pingcap/tidb/sessionctx/variable.(*SysVar).Validate()
      sessionctx/variable/variable.go:301 +0x12f
  github.com/pingcap/tidb/session.(*session).SetGlobalSysVar()
      session/session.go:1493 +0xe4
  github.com/pingcap/tidb/executor.(*SetExecutor).setSysVariable()
      executor/set.go:149 +0x6cd
  github.com/pingcap/tidb/executor.(*SetExecutor).Next()
      executor/set.go:99 +0xf59
  github.com/pingcap/tidb/executor.Next()
      executor/executor.go:328 +0x5c3
  github.com/pingcap/tidb/executor.(*ExecStmt).next()
      executor/adapter.go:1198 +0x89
  github.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelayExecutor()
      executor/adapter.go:950 +0x6f1
  github.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelay()
      executor/adapter.go:779 +0x34a
  github.com/pingcap/tidb/executor.(*ExecStmt).Exec()
      executor/adapter.go:574 +0x1299
  github.com/pingcap/tidb/session.runStmt()
      session/session.go:2351 +0x761
  github.com/pingcap/tidb/session.(*session).ExecuteStmt()
      session/session.go:2215 +0xfa5
  github.com/pingcap/tidb/testkit.(*TestKit).ExecWithContext()
      testkit/testkit.go:321 +0x96e
  github.com/pingcap/tidb/testkit.(*TestKit).MustExecWithContext()
      testkit/testkit.go:130 +0xb7
  github.com/pingcap/tidb/testkit.(*TestKit).MustExec()
      testkit/testkit.go:125 +0xf7
  executor/analyzetest/analyzetest_test.TestGlobalMemoryControlForAnalyze()
      executor/analyzetest/analyze_test.go:3072 +0xba
  github.com/pingcap/tidb/session.runStmt()
      session/session.go:2351 +0x761
  github.com/pingcap/tidb/session.(*session).ExecuteStmt()
      session/session.go:2215 +0xfa5
  github.com/pingcap/tidb/testkit.(*TestKit).ExecWithContext()
      testkit/testkit.go:321 +0x96e
  github.com/pingcap/tidb/testkit.(*TestKit).MustExecWithContext()
      testkit/testkit.go:130 +0xb7
  github.com/pingcap/tidb/testkit.(*TestKit).MustExec()
      testkit/testkit.go:125 +0xf7
  executor/analyzetest/analyzetest_test.TestGlobalMemoryControlForAnalyze()
      executor/analyzetest/analyze_test.go:3071 +0x9a
  github.com/pingcap/tidb/domain.(*Domain).rebuildSysVarCache()
      domain/sysvar_cache.go:146 +0x844
  github.com/pingcap/tidb/domain.(*Domain).LoadSysVarCacheLoop()
      domain/domain.go:1384 +0xa8
  github.com/pingcap/tidb/session.BootstrapSession()
      session/session.go:3337 +0x693
  github.com/pingcap/tidb/domain.(*Domain).GetSessionCache()
      domain/sysvar_cache.go:62 +0x59
  github.com/pingcap/tidb/session.(*session).loadCommonGlobalVariablesIfNeeded()
      session/session.go:3656 +0x104
  github.com/pingcap/tidb/session.(*session).ExecuteStmt()
      session/session.go:2134 +0x2a5
  github.com/pingcap/tidb/session.(*session).ExecuteInternal()
      session/session.go:1668 +0x471
  github.com/pingcap/tidb/domain.(*Domain).LoadPrivilegeLoop()
      domain/domain.go:1328 +0x130
  github.com/pingcap/tidb/session.BootstrapSession()
      session/session.go:3330 +0x644
  github.com/pingcap/tidb/testkit.bootstrap()
      testkit/mockstore.go:85 +0xac
  github.com/pingcap/tidb/testkit.CreateMockStoreAndDomain()
      testkit/mockstore.go:70 +0xe9
  executor/analyzetest/analyzetest_test.TestGlobalMemoryControlForAnalyze()
      executor/analyzetest/analyze_test.go:3068 +0x55
  testing.tRunner()
      GOROOT/src/testing/testing.go:1446 +0x216
  testing.(*T).Run.func1()
      GOROOT/src/testing/testing.go:1493 +0x47
==================

4. What is your TiDB version? (Required)
@hawkingrei hawkingrei added the type/bug The issue is confirmed as a bug. label Feb 1, 2023
@hawkingrei
Copy link
Member Author

https://prow.pingcap.net/view/gs/pingcapprow/logs/tidb_race/1620631429853483008

@chrysan chrysan self-assigned this Feb 1, 2023
@ti-chi-bot ti-chi-bot added may-affects-4.0 This bug maybe affects 4.0.x versions. may-affects-5.0 This bug maybe affects 5.0.x versions. may-affects-5.1 This bug maybe affects 5.1.x versions. may-affects-5.2 This bug maybe affects 5.2.x versions. may-affects-5.3 This bug maybe affects 5.3.x versions. may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-6.0 may-affects-6.1 may-affects-6.2 may-affects-6.3 may-affects-6.4 may-affects-6.5 labels Feb 2, 2023
@ChenPeng2013 ChenPeng2013 removed may-affects-4.0 This bug maybe affects 4.0.x versions. may-affects-5.1 This bug maybe affects 5.1.x versions. may-affects-5.2 This bug maybe affects 5.2.x versions. may-affects-5.3 This bug maybe affects 5.3.x versions. may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-5.0 This bug maybe affects 5.0.x versions. may-affects-6.0 may-affects-6.1 may-affects-6.2 may-affects-6.3 may-affects-6.4 may-affects-6.5 labels Feb 2, 2023
@chrysan chrysan linked a pull request Feb 23, 2023 that will close this issue
Open
12 tasks
@hawkingrei
Copy link
Member Author

https://do.pingcap.net/jenkins/blue/organizations/jenkins/pingcap%2Ftidb%2Fghpr_unit_test/detail/ghpr_unit_test/17584/pipeline

@hawkingrei
Copy link
Member Author

It is still in existed.

@hawkingrei hawkingrei mentioned this issue Feb 23, 2023
Open
@chrysan chrysan mentioned this issue Jun 2, 2023
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Jun 2, 2023
@chrysan
executor: make prepare analyze killed globally instead of query quota (…
d5861ef 
…#44352)

ref #40946, close #44320
This was referenced Jun 2, 2023
Merged
Merged
ti-chi-bot bot pushed a commit that referenced this issue Jun 5, 2023
@ti-chi-bot
executor: make prepare analyze killed globally instead of query quota (…
b768eb5 
…#44352) (#44367)

ref #40946, close #44320
@jebter jebter added the sig/planner SIG: Planner label Sep 11, 2023
@fixdb fixdb assigned hawkingrei and unassigned chrysan Feb 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hawkingrei hawkingrei

Labels
affects-6.6 affects-7.0 affects-7.1 affects-7.5 affects-8.1 component/test severity/moderate sig/planner SIG: Planner type/bug The issue is confirmed as a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

util: fix datarace and leak in TestGlobalMemoryControlForAnalyze chrysan/tidb
7 participants
@hawkingrei @jebter @chrysan @ChenPeng2013 @qw4990 @VelocityLight @ti-chi-bot