Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Panic: concurrent map writes of ParallelNestedLoopApplyExec's SimpleLRUCache #50347

Closed
yibin87 opened this issue Jan 12, 2024 · 15 comments · Fixed by #50356
Closed

Panic: concurrent map writes of ParallelNestedLoopApplyExec's SimpleLRUCache #50347

yibin87 opened this issue Jan 12, 2024 · 15 comments · Fixed by #50356
Assignees
Labels
affects-5.4 This bug affects the 5.4.x(LTS) versions. affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.5 This bug affects the 7.5.x(LTS) versions. found/gs found by gs report/customer Customers have encountered this bug. severity/major sig/execution SIG execution type/bug The issue is confirmed as a bug.

Comments

@yibin87
Copy link
Contributor

yibin87 commented Jan 12, 2024

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

2. What did you expect to see? (Required)

3. What did you see instead (Required)

From customer case, not sure how to reproduce it, just the panic stack
fatal error: concurrent map writes

goroutine 349945042302 [running]:
github.com/pingcap/tidb/util/kvcache.(*SimpleLRUCache).Put(0xc6eb5946c0, {0x5d72e00, 0xc82454dab8}, {0x52a62a0?, 0xc1d757f730})
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/util/kvcache/simple_lru.go:114 +0x396
github.com/pingcap/tidb/executor.(*applyCache).put(0xcabeb98aa0, {0xc1c8be0060?, 0xb12eb0f14f0b7?, 0x8a12c80?}, {0x52a62a0, 0xc1d757f730})
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/apply_cache.go:66 +0xe5
github.com/pingcap/tidb/executor.(*applyCache).Set(0xcabeb98aa0, {0xc1c8be0060, 0x2e, 0x2e}, 0xc1d757f730)
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/apply_cache.go:93 +0x132
github.com/pingcap/tidb/executor.(*ParallelNestedLoopApplyExec).fetchAllInners(0xc617193d40, {0x5d98390, 0xc60643b440}, 0x9)
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/parallel_apply.go:338 +0xa8f
github.com/pingcap/tidb/executor.(*ParallelNestedLoopApplyExec).fillInnerChunk(0xc617193d40, {0x5d98390, 0xc60643b440}, 0x9, 0xc3bde2ebe0)
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/parallel_apply.go:384 +0x32f
github.com/pingcap/tidb/executor.(*ParallelNestedLoopApplyExec).innerWorker(0xc617193d40, {0x5d98390, 0xc60643b440}, 0xc713dd9e60?)
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/parallel_apply.go:247 +0x1ba
created by github.com/pingcap/tidb/executor.(*ParallelNestedLoopApplyExec).Next
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/parallel_apply.go:146 +0x328
tidb_stderr.log

4. What is your TiDB version? (Required)

v7.1.1

@yibin87 yibin87 added the type/bug The issue is confirmed as a bug. label Jan 12, 2024
@jackysp jackysp added the found/gs found by gs label Jan 12, 2024
@SeaRise
Copy link
Contributor

SeaRise commented Jan 12, 2024

/cc @SeaRise

@SeaRise
Copy link
Contributor

SeaRise commented Jan 12, 2024

/assign @SeaRise

@SeaRise
Copy link
Contributor

SeaRise commented Jan 12, 2024

evictedKey, evictedValue, evicted := c.cache.RemoveOldest()

seems lock should be added here.

@SeaRise
Copy link
Contributor

SeaRise commented Jan 12, 2024

/sig compute

Copy link

ti-chi-bot bot commented Jan 12, 2024

@SeaRise: The label(s) sig/compute cannot be applied, because the repository doesn't have them.

In response to this:

/sig compute

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@SeaRise
Copy link
Contributor

SeaRise commented Jan 12, 2024

/sig execution

@ti-chi-bot ti-chi-bot bot added the sig/execution SIG execution label Jan 12, 2024
@SeaRise
Copy link
Contributor

SeaRise commented Jan 12, 2024

/label affects-7.5

@ti-chi-bot ti-chi-bot bot added the affects-7.5 This bug affects the 7.5.x(LTS) versions. label Jan 12, 2024
@SeaRise
Copy link
Contributor

SeaRise commented Jan 12, 2024

/label affects-7.1

@ti-chi-bot ti-chi-bot bot added the affects-7.1 This bug affects the 7.1.x(LTS) versions. label Jan 12, 2024
@SeaRise
Copy link
Contributor

SeaRise commented Jan 12, 2024

/label affects-6.5

@ti-chi-bot ti-chi-bot bot added the affects-6.5 This bug affects the 6.5.x(LTS) versions. label Jan 12, 2024
@SeaRise
Copy link
Contributor

SeaRise commented Jan 12, 2024

/label affects-6.1

@ti-chi-bot ti-chi-bot bot added the affects-6.1 This bug affects the 6.1.x(LTS) versions. label Jan 12, 2024
@SeaRise
Copy link
Contributor

SeaRise commented Jan 12, 2024

/label affects-5.4

@ti-chi-bot ti-chi-bot bot added the affects-5.4 This bug affects the 5.4.x(LTS) versions. label Jan 12, 2024
@SeaRise
Copy link
Contributor

SeaRise commented Jan 12, 2024

/label affects-5.3

Copy link

ti-chi-bot bot commented Jan 12, 2024

@SeaRise: The label(s) affects-5.3 cannot be applied. These labels are supported: fuzz/sqlancer, challenge-program, compatibility-breaker, first-time-contributor, contribution, good first issue, correctness, duplicate, proposal, security, ok-to-test, needs-ok-to-test, needs-more-info, needs-cherry-pick-release-5.4, needs-cherry-pick-release-6.1, needs-cherry-pick-release-6.5, needs-cherry-pick-release-7.1, needs-cherry-pick-release-7.5, needs-cherry-pick-release-7.6, affects-5.4, affects-6.1, affects-6.5, affects-7.1, affects-7.5, affects-7.6, may-affects-5.4, may-affects-6.1, may-affects-6.5, may-affects-7.1, may-affects-7.5, may-affects-7.6.

In response to this:

/label affects-5.3

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@SeaRise
Copy link
Contributor

SeaRise commented Jan 12, 2024

/severity major

@seiya-annie
Copy link

/found customer

@ti-chi-bot ti-chi-bot bot added the report/customer Customers have encountered this bug. label Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-5.4 This bug affects the 5.4.x(LTS) versions. affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.5 This bug affects the 7.5.x(LTS) versions. found/gs found by gs report/customer Customers have encountered this bug. severity/major sig/execution SIG execution type/bug The issue is confirmed as a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants