Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Proposal for vector db semantic convention #1231
base: main
Are you sure you want to change the base?
Proposal for vector db semantic convention #1231
Changes from 3 commits
fa8ee3070687205e12a863b61784828bacc53d82d4a3330ffe5ff387da6649bd99ec1061b0f2cceae2ca6db7ec5e5e0d9dae0e06603b67bf93d2cbef411554daa0a14bc8a63ca5f8661d996cd9a10e75ffd0f2e71c6bd009feb74d235776681dca47ff03da1523bcb9fd891f6bc2ddb1fc90f3f24cc8123d50dc106a67d4949b198a88ac32df19d7694c2ca1210ecb9765a4a8ccd11f74def570File filter
Filter by extension
Conversations
Jump to
There are no files selected for viewing
Check failure on line 556 in model/registry/db.yaml
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should come up with a more common attribute not specific to vector dbs.
Many databases allow to limit number of returned rows:
Suggesting
db.query.max_returned_items. The actual returned count could be even better -db.query.item_countcould mean items inserted or returned depending on the operation.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lmolkova I see the similarity here but I think the
db.vector.query.top-kis more specific from a semantic point of view and more related to vectors, since it specifies the top k results in order, starting from the most similar. In semantic search we have this similarity value that is always present in any result that we don't have in standard database. Thelimitparameter of SQL returns the first k results but not in order, it depends on how you build the query (e.g. usingORDER BY).I personally think we should keep
db.vector.query.top-kand potentially add adb.query.limit(ordb.query.max_returned_itemsas you suggested) in a separate PR.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Out of the few dbs I checked, they use
limitin vector searchlimit- same with cosmos and other sql databasesSo we're saying that DB instrumentations will need to detect if query is related to vector search or not and depending on this populate top-k or limit. That's difficult or impossible, but most importantly inconsistent and depends on instrumentation capabilities.
I.e. instrumentations that don't have vector-db specifics and those that do will use different attributes for the same thing.
So, I'd still prefer
db.query.limitor something similar (and it should be under the same condition asdb.query.text- we cannot require instrumentations to do query parsing)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lmolkova do you agree that
top-kandlimitare two different concepts, based on my previous comment? If they are I think we cannot use a single attribute (e.g.db.query.limit) to manage both.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lmolkova just a reminder for this, thanks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ezimuel I see that databases use both terms to describe the same thing (see my comment above).
Let's say you have a postgres query like
SELECT * FROM items ORDER BY embedding <-> '[3,1,2]' LIMIT 5;- the general-purpose DB instrumentation can reportlimit. If you make it understand vector search syntax, it may be able to usetop_kinstead, but that's would be inconsistent and unfamiliar for those who use vector search in postgres.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lmolkova I see your point but I think
top-khas a different meaning fromlimit. If you are using a relation database as vector db,limitis fine since you are building an SQL statement and you specify the order. But, if you are using a native vector database (e.g. Qdrant), thetop-kis more relevant sincetopimplies the order, using a similarity metric.I think we should add both:
db.query.limitdb.vector.query.top-kThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lmolkova I found that the
SELECT LIMITis not part of the SQL standard. A statement compliant with SQL standard isFETCH FIRST. Moreover, I discovered that we already havedb.operation.parameter,<key>in the semantic convention. This means, we can havedb.operation.parameter.fetch-first. I don't think we need to add alimitorfetch-firstindb.query. What do you think?At the same time, I believe we should include
db.vector.query.top-k. As noted earlier, it differs in meaning fromfetch-firstand, additionally, this parameter is much easier to retrieve in a vector database because it's not part of any statement.Check failure on line 595 in model/registry/db.yaml