Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhance: Add secondary index for querynode segment manager #38311

Conversation

congqixia
Copy link
Contributor

Related to #37630

Add secondary index with vchannel to reduce GetBy rlock holding time when segment number is large.

Related to milvus-io#37630

Add secondary index with vchannel to reduce `GetBy` rlock holding time
when segment number is large.

Signed-off-by: Congqi Xia <[email protected]>
@sre-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: congqixia

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sre-ci-robot sre-ci-robot added approved size/XL Denotes a PR that changes 500-999 lines. labels Dec 9, 2024
@mergify mergify bot added dco-passed DCO check passed. kind/enhancement Issues or changes related to enhancement labels Dec 9, 2024
@congqixia congqixia added the 10kcp PR related to 10k collection partitions Project label Dec 9, 2024
Copy link

codecov bot commented Dec 9, 2024

Codecov Report

Attention: Patch coverage is 83.18584% with 38 lines in your changes missing coverage. Please review.

Project coverage is 81.04%. Comparing base (7a5aea1) to head (a7c88f8).
Report is 4 commits behind head on master.

Files with missing lines Patch % Lines
internal/querynodev2/segments/segment_filter.go 62.71% 21 Missing and 1 partial ⚠️
internal/querynodev2/segments/manager.go 90.30% 10 Missing and 6 partials ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #38311      +/-   ##
==========================================
- Coverage   81.08%   81.04%   -0.05%     
==========================================
  Files        1372     1373       +1     
  Lines      191671   191704      +33     
==========================================
- Hits       155412   155359      -53     
- Misses      30762    30833      +71     
- Partials     5497     5512      +15     
Components Coverage Δ
Client 74.58% <ø> (ø)
Core 68.83% <ø> (ø)
Go 83.15% <83.18%> (-0.06%) ⬇️
Files with missing lines Coverage Δ
pkg/util/metautil/channel.go 88.00% <100.00%> (+0.32%) ⬆️
internal/querynodev2/segments/manager.go 77.92% <90.30%> (+1.81%) ⬆️
internal/querynodev2/segments/segment_filter.go 62.71% <62.71%> (ø)

... and 33 files with indirect coverage changes

czs007 pushed a commit that referenced this pull request Dec 9, 2024
…38312)

Cherry pick from pr
#38311
Related to #37630

Add secondary index with vchannel to reduce `GetBy` rlock holding time
when segment number is large.

Signed-off-by: Congqi Xia <[email protected]>
@mergify mergify bot added the ci-passed label Dec 9, 2024
@tedxu
Copy link
Contributor

tedxu commented Dec 10, 2024

/lgtm

@sre-ci-robot sre-ci-robot merged commit b8e3795 into milvus-io:master Dec 10, 2024
19 of 20 checks passed
congqixia added a commit to congqixia/milvus that referenced this pull request Dec 10, 2024
…#38311)

Related to milvus-io#37630

Add secondary index with vchannel to reduce `GetBy` rlock holding time
when segment number is large.

Signed-off-by: Congqi Xia <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
10kcp PR related to 10k collection partitions Project approved ci-passed dco-passed DCO check passed. kind/enhancement Issues or changes related to enhancement lgtm size/XL Denotes a PR that changes 500-999 lines.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants