Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

lightning report "received message larger than max (18552234 vs. 4194304)" during duplicate resolution #56114

Closed
D3Hunter opened this issue Sep 18, 2024 · 2 comments · Fixed by #56771
Closed

lightning report "received message larger than max (18552234 vs. 4194304)" during duplicate resolution #56114

D3Hunter opened this issue Sep 18, 2024 · 2 comments · Fixed by #56771
Assignees
@fishiu
Labels
affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.5 This bug affects the 7.5.x(LTS) versions. affects-8.1 This bug affects the 8.1.x(LTS) versions. affects-8.5 This bug affects the 8.5.x(LTS) versions. component/lightning This issue is related to Lightning of TiDB. good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. severity/major type/bug The issue is confirmed as a bug.

Comments

@D3Hunter
Copy link
Contributor

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

currently, we use default max-message-size which is 4M, see

tidb/pkg/lightning/backend/local/local.go

Line 144 in bda79ca

func (f *importClientFactoryImpl) makeConn(ctx context.Context, storeID uint64) (*grpc.ClientConn, error) {

2. What did you expect to see? (Required)

no error

3. What did you see instead (Required)

err

4. What is your TiDB version? (Required)

8.1.0
@D3Hunter D3Hunter added type/bug The issue is confirmed as a bug. severity/major component/lightning This issue is related to Lightning of TiDB. affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.5 This bug affects the 7.5.x(LTS) versions. affects-8.1 This bug affects the 8.1.x(LTS) versions. labels Sep 18, 2024
@ti-chi-bot ti-chi-bot bot added may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-6.1 may-affects-6.5 labels Sep 18, 2024
@D3Hunter
Copy link
Contributor Author

tikv send max 4k KV on each response. each KV is about 4K+ in the issue case
https://github.com/tikv/tikv/blob/076b79c54b9e7656f9119865d4ed75bec3bd58d6/src/import/duplicate_detect.rs#L16

@D3Hunter D3Hunter added good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. and removed good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. labels Sep 18, 2024
@fishiu
Copy link
Contributor

fishiu commented Sep 24, 2024

/assign

@fishiu fishiu mentioned this issue Oct 22, 2024
Merged
13 tasks
@lance6716 lance6716 added affects-5.3 This bug affects 5.3.x versions. affects-5.4 This bug affects the 5.4.x(LTS) versions. affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-6.5 This bug affects the 6.5.x(LTS) versions. labels Oct 29, 2024
@ti-chi-bot ti-chi-bot bot added affects-8.5 This bug affects the 8.5.x(LTS) versions. and removed may-affects-6.1 may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-6.5 labels Oct 29, 2024
@D3Hunter D3Hunter removed affects-5.3 This bug affects 5.3.x versions. affects-5.4 This bug affects the 5.4.x(LTS) versions. labels Nov 5, 2024
@ti-chi-bot ti-chi-bot bot closed this as completed in b522e98 Nov 6, 2024
ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this issue Nov 11, 2024
@fishiu @ti-chi-bot
lightning: unlimited grpc MaxCallRecvMsgSize for TikvImporter (pingca…
07e3783 
…p#56771)

close pingcap#56114
@ti-chi-bot ti-chi-bot mentioned this issue Nov 11, 2024
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Nov 11, 2024
@ti-chi-bot
lightning: unlimited grpc MaxCallRecvMsgSize for TikvImporter (#56771) (
1e0d6e3 
#57263)

close #56114
@ti-chi-bot ti-chi-bot mentioned this issue Nov 13, 2024
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Nov 19, 2024
@ti-chi-bot
lightning: unlimited grpc MaxCallRecvMsgSize for TikvImporter (#56771) (
238f8a8 
#57346)

close #56114
ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this issue Dec 3, 2024
@fishiu @ti-chi-bot
lightning: unlimited grpc MaxCallRecvMsgSize for TikvImporter (pingca…
729f136 
…p#56771)

close pingcap#56114
@ti-chi-bot ti-chi-bot mentioned this issue Dec 3, 2024
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Dec 11, 2024
@ti-chi-bot
lightning: unlimited grpc MaxCallRecvMsgSize for TikvImporter (#56771) (
7da37ba 
#57920)

close #56114
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@fishiu fishiu

Labels
affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.5 This bug affects the 7.5.x(LTS) versions. affects-8.1 This bug affects the 8.1.x(LTS) versions. affects-8.5 This bug affects the 8.5.x(LTS) versions. component/lightning This issue is related to Lightning of TiDB. good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. severity/major type/bug The issue is confirmed as a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

lightning: unlimited grpc MaxCallRecvMsgSize for TikvImporter fishiu/tidb
3 participants
@lance6716 @D3Hunter @fishiu