Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Enhancement]: Optimize clustering compaction #34495

Closed
1 task done
wayblink opened this issue Jul 8, 2024 · 3 comments
Closed
1 task done

[Enhancement]: Optimize clustering compaction #34495

wayblink opened this issue Jul 8, 2024 · 3 comments
Assignees
@wayblink @xiaocai2333
Labels
kind/enhancement Issues or changes related to enhancement stale indicates no udpates for 30 days

Comments

@wayblink
Copy link
Contributor

wayblink commented Jul 8, 2024

Is there an existing issue for this?

  • I have searched the existing issues

What would you like to be added?

major compaction is generally completed but still have many optimization points

Why is this needed?

No response

Anything else?

No response
@wayblink wayblink added the kind/enhancement Issues or changes related to enhancement label Jul 8, 2024
@wayblink
Copy link
Contributor Author

wayblink commented Jul 8, 2024

/assign

@wayblink wayblink mentioned this issue Jul 9, 2024
Closed
1 task
@wayblink wayblink changed the title [Enhancement]: Optimize major compaction [Enhancement]: Optimize clustering compaction Jul 9, 2024
@wayblink wayblink mentioned this issue Jul 9, 2024
Closed
1 task
@xiaocai2333
Copy link
Contributor

/assign

This was referenced Jul 24, 2024
Merged
Merged
sre-ci-robot pushed a commit that referenced this issue Jul 24, 2024
@xiaocai2333
enhance: [cherry-pick] Refine code for GetRecoveryInfo (#34974)
12a24c3 
issue: #34495 

master pr: #34973

Signed-off-by: Cai Zhang <[email protected]>
sre-ci-robot pushed a commit that referenced this issue Jul 28, 2024
@xiaocai2333
enhance: Refine code for GetRecoveryInfo (#34973)
497afcb 
issue: #34495

Signed-off-by: Cai Zhang <[email protected]>
This was referenced Jul 29, 2024
Merged
Merged
Merged
Merged
sre-ci-robot pushed a commit that referenced this issue Jul 31, 2024
@xiaocai2333
enhance: [cherry-pick] Skip updating the level that are already at th…
4bbb7b8 
…e specified level (#35093)

issue: #34495 

master pr: #35094

Signed-off-by: Cai Zhang <[email protected]>
sre-ci-robot pushed a commit that referenced this issue Jul 31, 2024
@xiaocai2333
enhance: [cherry-pick] Change the fixed value to a ratio for clusteri…
c340f38 
…ng segment size (#35075)

issue: #34495 

master pr: #35076

Signed-off-by: Cai Zhang <[email protected]>
This was referenced Jul 31, 2024
Merged
Merged
Merged
sre-ci-robot pushed a commit that referenced this issue Aug 1, 2024
@xiaocai2333
enhance: Skip updating the level that are already at the specified le…
089fb2b 
…vel (#35094)

issue: #34495

Signed-off-by: Cai Zhang <[email protected]>
sre-ci-robot pushed a commit that referenced this issue Aug 1, 2024
@xiaocai2333
fix: Fix data race for clustering buffer writer (#35145)
9412002 
issue: #34495

Signed-off-by: Cai Zhang <[email protected]>
This was referenced Aug 1, 2024
Merged
Merged
sre-ci-robot pushed a commit that referenced this issue Aug 1, 2024
@xiaocai2333
fix: [cherry-pick] Fix data race for clustering buffer writer (#35146)
f718410 
issue: #34495 

master pr: #35145

Signed-off-by: Cai Zhang <[email protected]>
sre-ci-robot pushed a commit that referenced this issue Aug 1, 2024
@xiaocai2333
enhance: Change the fixed value to a ratio for clustering segment size (
196a798 
#35076)

issue: #34495

---------

Signed-off-by: Cai Zhang <[email protected]>
sre-ci-robot pushed a commit that referenced this issue Aug 2, 2024
@xiaocai2333
enhance:[cherry-pick] Support calculate segment maxRows with diskann (#…
5f3d41d 
…35155)

issue: #34495 

master pr: #35076

Signed-off-by: Cai Zhang <[email protected]>
sre-ci-robot pushed a commit that referenced this issue Aug 2, 2024
@wayblink
enhance: [cherry-pick] unify time in clustering compaction task to un…
8137c26 
…ix (#35168)

related: #34495 
pr: #35167

Signed-off-by: wayblink <[email protected]>
sre-ci-robot pushed a commit that referenced this issue Aug 2, 2024
@wayblink
enhance: unify time in clustering compaction task to unix (#35167)
9546266 
#34495

Signed-off-by: wayblink <[email protected]>
sumitd2 pushed a commit to sumitd2/milvus that referenced this issue Aug 6, 2024
@xiaocai2333 @sumitd2
enhance: Refine code for GetRecoveryInfo (milvus-io#34973)
a031fe7 
issue: milvus-io#34495

Signed-off-by: Cai Zhang <[email protected]>
Signed-off-by: Sumit Dubey <[email protected]>
sumitd2 pushed a commit to sumitd2/milvus that referenced this issue Aug 6, 2024
@xiaocai2333 @sumitd2
enhance: Skip updating the level that are already at the specified le…
dcabc1b 
…vel (milvus-io#35094)

issue: milvus-io#34495

Signed-off-by: Cai Zhang <[email protected]>
Signed-off-by: Sumit Dubey <[email protected]>
sumitd2 pushed a commit to sumitd2/milvus that referenced this issue Aug 6, 2024
@xiaocai2333 @sumitd2
fix: Fix data race for clustering buffer writer (milvus-io#35145)
d72b4dd 
issue: milvus-io#34495

Signed-off-by: Cai Zhang <[email protected]>
Signed-off-by: Sumit Dubey <[email protected]>
sumitd2 pushed a commit to sumitd2/milvus that referenced this issue Aug 6, 2024
@xiaocai2333 @sumitd2
enhance: Change the fixed value to a ratio for clustering segment size (
8dc2603 
milvus-io#35076)

issue: milvus-io#34495

---------

Signed-off-by: Cai Zhang <[email protected]>
Signed-off-by: Sumit Dubey <[email protected]>
sumitd2 pushed a commit to sumitd2/milvus that referenced this issue Aug 6, 2024
@wayblink @sumitd2
enhance: unify time in clustering compaction task to unix (milvus-io#…
8b30b67 
…35167)

milvus-io#34495

Signed-off-by: wayblink <[email protected]>
Signed-off-by: Sumit Dubey <[email protected]>
@stale Stale
Copy link

stale bot commented Aug 24, 2024

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

@stale stale bot added the stale indicates no udpates for 30 days label Aug 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@wayblink wayblink

@xiaocai2333 xiaocai2333

Labels
kind/enhancement Issues or changes related to enhancement stale indicates no udpates for 30 days
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@wayblink @xiaocai2333