Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Enhancement]: Decouple Milvus's dependence on vector index types #34298

Closed
1 task done
foxspy opened this issue Jul 1, 2024 · 8 comments
Closed
1 task done

[Enhancement]: Decouple Milvus's dependence on vector index types #34298

foxspy opened this issue Jul 1, 2024 · 8 comments
Assignees
@foxspy
Labels
kind/enhancement Issues or changes related to enhancement stale indicates no udpates for 30 days

Comments

@foxspy
Copy link
Contributor

foxspy commented Jul 1, 2024

Is there an existing issue for this?

  • I have searched the existing issues

What would you like to be added?

In Milvus, some code heavily depends on the types of vector indices. This behavior is unreasonable, as Milvus does not grasp the specific differences between different indices. If there are indeed corresponding needs, the index engine (Knowhere) should provide the appropriate interfaces to fulfill them. With the increasing diversity of vector indices, this decoupling needs to be addressed as soon as possible.

Why is this needed?

No response

Anything else?

No response
@foxspy foxspy added the kind/enhancement Issues or changes related to enhancement label Jul 1, 2024
@foxspy
Copy link
Contributor Author

foxspy commented Jul 1, 2024

/assign @foxspy

@stale Stale
Copy link

stale bot commented Aug 4, 2024

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

@stale stale bot added the stale indicates no udpates for 30 days label Aug 4, 2024
@stale stale bot closed this as completed Aug 11, 2024
@foxspy
Copy link
Contributor Author

foxspy commented Aug 28, 2024

/reopen

@sre-ci-robot sre-ci-robot reopened this Aug 28, 2024
@sre-ci-robot
Copy link
Contributor

@foxspy: Reopened this issue.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@stale stale bot removed the stale indicates no udpates for 30 days label Aug 28, 2024
This was referenced Aug 28, 2024
Closed
Closed
@foxspy foxspy mentioned this issue Sep 18, 2024
Closed
@stale Stale
Copy link

stale bot commented Sep 29, 2024

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

@stale stale bot added the stale indicates no udpates for 30 days label Sep 29, 2024
@foxspy
Copy link
Contributor Author

foxspy commented Oct 11, 2024

/reopen

@foxspy
Copy link
Contributor Author

foxspy commented Oct 11, 2024 •
edited
Loading

There should be some independent work to decouple knowhere with milvus

  • Unify the conditional logic based on vector index type into Knowhere
  • Unify the config check logic for vector index type into Knowhere
  • Unify the common vector parameter filling logic
  • Adapt for the diskann parameter filling logic
  • Adapt for the autoIndex parameter filling logic
  • Clear useless checker code
  • Disable hnsw binary
  • Remove useless CGo function from knowhere

@stale stale bot removed the stale indicates no udpates for 30 days label Oct 11, 2024
This was referenced Oct 14, 2024
Merged
Merged
Merged
sre-ci-robot pushed a commit that referenced this issue Oct 17, 2024
@foxspy
enhance: add vector index mgr to remove vector index type dependency (#…
3de57ec 
…36843)

issue: #34298

Signed-off-by: xianliang.li <[email protected]>
sre-ci-robot pushed a commit that referenced this issue Oct 28, 2024
@foxspy
enhance: add an unify vector index config checker (#36844)
d7b2ffe 
issue: #34298

Signed-off-by: xianliang.li <[email protected]>
This was referenced Oct 31, 2024
Merged
Merged
Merged
sre-ci-robot pushed a commit that referenced this issue Oct 31, 2024
@foxspy
enhance: add unify vector index config management (#36846)
3224e58 
issue: #34298

Signed-off-by: xianliang.li <[email protected]>
sre-ci-robot pushed a commit that referenced this issue Oct 31, 2024
@foxspy
enhance: remove useless vector index param checker (#37329)
642a651 
issue: #34298 
because all vector index config checker has been moved into
vector_index_checker, then the useless checkers can be removed.

Signed-off-by: xianliang.li <[email protected]>
sre-ci-robot pushed a commit that referenced this issue Nov 1, 2024
@foxspy
enhance: [CP25]add unify vector index config management (#37331)
5aee184 
issue: #34298 
pr: #36844

Signed-off-by: xianliang.li <[email protected]>
sre-ci-robot pushed a commit that referenced this issue Nov 4, 2024
@foxspy
enhance: [CP25] remove useless vector index param checker (#37330)
8ba0a4a 
issue: #34298 
pr: #37329

Signed-off-by: xianliang.li <[email protected]>
This was referenced Nov 4, 2024
Merged
Merged
sre-ci-robot pushed a commit that referenced this issue Nov 4, 2024
@foxspy
fix: process null value in yaml (#37418)
1b98bb4 
issue: #34298 
fix key: null defined in the yaml file. 
viper will parse it as "", and yaml v3 will parse it as "null".

Signed-off-by: xianliang.li <[email protected]>
sre-ci-robot pushed a commit that referenced this issue Nov 11, 2024
@foxspy
fix: use yaml.v2 as yaml parser (#37423)
81141bd 
issue: #34298 
Viper uses yaml.v2 as the parser. This PR will adopt the parsing logic
from Viper to handle YAML files, ensuring maximum consistency in
parsing.

Signed-off-by: xianliang.li <[email protected]>
@stale Stale
Copy link

stale bot commented Dec 21, 2024

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

@stale stale bot added the stale indicates no udpates for 30 days label Dec 21, 2024
@stale stale bot closed this as completed Dec 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@foxspy foxspy

Labels
kind/enhancement Issues or changes related to enhancement stale indicates no udpates for 30 days
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@foxspy @sre-ci-robot