From 7376189b252e5db738b840c1c9d2de08ff409280 Mon Sep 17 00:00:00 2001 From: Maria Craig Date: Mon, 31 Jul 2023 15:36:37 +0200 Subject: [PATCH] Release v1.3.0 (#244) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Bump openapi.yaml file * Facet Search API (#246) * Init specification * Adjust spec files to PR number * Fix content-type open-api * Remove fix content-type * Add the telemetry * Update text/0246-facet-search-api.md Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com> * Update text/0034-telemetry-policies.md --------- Co-authored-by: Clément Renault Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com> * Faceting Setting API - sortFacetValuesBy (#247) * init spec * Add property description and open-api.yaml * Removes future possibility * Update the telemetry * Update the error of the sortFacetValuesBy setting --------- Co-authored-by: Clément Renault * Vector Search - `EXPERIMENTAL` (#248) * Init spec * Fix the vector store fields * Add an information on the invalid_search_vector error code * Add an information on the invalid_vectors_field error codes * Define the new max_vector_size analytic * Update the open-api file with vector capabilities * Apply suggestions from code review * Update open-api.yaml Co-authored-by: Maria Craig * Update open-api.yaml Co-authored-by: Maria Craig * Update text/0118-search-api.md Co-authored-by: Maria Craig * Update text/0061-error-format-and-definitions.md Co-authored-by: Maria Craig --------- Co-authored-by: Kerollmops Co-authored-by: Maria Craig * Tasks route total (#253) * Update the tasks route info to add the total field * Update the OpenApi of the tasks route * Document the latest metrics improvements (#242) * document the latest metrics improvements * Update text/0174-metrics-api.md Co-authored-by: Guillaume Mourier * Update text/0174-metrics-api.md Co-authored-by: Guillaume Mourier * adds some details on the meilisearch_db_size metrics * update the open api metrics smaple * update a bad copy paste * Update open-api.yaml * prefix prometheus metrics by meilisearch_ * Introduce two new metrics --------- Co-authored-by: Guillaume Mourier Co-authored-by: Kerollmops --------- Co-authored-by: Guillaume Mourier Co-authored-by: Clément Renault Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com> Co-authored-by: Clément Renault Co-authored-by: Tamo --- open-api.yaml | 506 ++++++++++++++++++++-- text/0034-telemetry-policies.md | 60 ++- text/0060-tasks-api.md | 4 + text/0061-error-format-and-definitions.md | 201 ++++++++- text/0085-api-keys.md | 2 + text/0118-search-api.md | 183 ++++++-- text/0124-documents-api.md | 124 +++--- text/0174-metrics-api.md | 101 ++++- text/0193-experimental-features.md | 98 +++++ text/0194-experimental-feature-api.md | 55 +++ text/0195-ranking-score.md | 183 ++++++++ text/0246-facet-search-api.md | 150 +++++++ text/157-faceting-setting-api.md | 40 +- 13 files changed, 1549 insertions(+), 158 deletions(-) create mode 100644 text/0193-experimental-features.md create mode 100644 text/0194-experimental-feature-api.md create mode 100644 text/0195-ranking-score.md create mode 100644 text/0246-facet-search-api.md diff --git a/open-api.yaml b/open-api.yaml index aeadaa3d..7feefb93 100644 --- a/open-api.yaml +++ b/open-api.yaml @@ -2,7 +2,7 @@ openapi: 3.1.0 info: title: Meilisearch Core API description: 'Search documents, configure and manage the Meilisearch engine.' - version: 1.2.0 + version: 1.3.0 contact: name: Meilisearch email: bonjour@Meilisearch.com @@ -117,6 +117,157 @@ components: - start - length description: Starting position and length in bytes of the matched term in the returned value + order: + type: integer + description: The order that this ranking rule was applied + customRankingRuleDetails: + type: object + properties: + order: + $ref: '#/components/schemas/order' + value: + type: + - string + - number + - point + description: The value that was used for sorting this document + distance: + type: number + description: The distance between the target point and the geoPoint in the document + required: + - order + - value + description: Custom rule in the form of either `attribute:direction` or `_geoPoint(lat, lng):direction`. + score: + type: number + description: | + The relevancy score of a document according to a ranking rule and relative to a search query. Higher is better. + + `1.0` indicates a perfect match, `0.0` no match at all (Meilisearch should not return documents that don't match the query). + rankingScoreDetails: + type: object + properties: + words: + type: object + properties: + order: + $ref: '#/components/schemas/order' + matchingWords: + type: integer + description: the number of words from the query found + maxMatchingWords: + type: integer + score: + $ref: '#/components/schemas/score' + required: + - order + - matchingWords + - maxMatchingWords + - score + typo: + type: object + properties: + order: + $ref: '#/components/schemas/order' + typoCount: + type: integer + description: The number of typos to correct in the query to match that document. + maxTypoCount: + type: integer + description: The maximum number of typos that can be corrected in the query to match a document. + score: + $ref: '#/components/schemas/score' + required: + - order + - typoCount + - maxTypoCount + - score + proximity: + type: object + properties: + order: + $ref: '#/components/schemas/order' + score: + $ref: '#/components/schemas/score' + required: + - order + - score + attribute: + type: object + properties: + order: + $ref: '#/components/schemas/order' + attribute_ranking_order_score: + type: number + description: | + Score computed depending on the first attribute each word of the query appears in. + + The first attribute in the `searchableAttributes` list yields the highest score, the last attribute the lowest. + query_word_distance_score: + type: number + description: | + Score computed depending on the position the attributes where each word of the query appears in. + + Words appearing in an attribute at the same position as in the query yield the highest score. The greater the distance to the position + in the query, the lower the score. + score: + $ref: '#/components/schemas/score' + required: + - order + - attribute_ranking_order_score + - query_word_distance_score + - score + exactness: + type: object + properties: + order: + $ref: '#/components/schemas/order' + matchType: + type: string + description: | + One of `exactMatch`, `matchesStart` or `noExactMatch`. + - `exactMatch`: the document contains an attribute that exactly matches the query. + - `matchesStart`: the document contains an attribute that exactly starts with the query. + - `noExactMatch`: any other document. + score: + $ref: '#/components/schemas/score' + required: + - order + - matchType + - score + additionalProperties: + $ref: '#/components/schemas/customRankingRuleDetails' + description: (EXPERIMENTAL) The ranking score per ranking rule. + examples: + With sort: + words: + order: 0 + matchingWords: 7 + maxMatchingWords: 7 + score: 1.0 + "typo": + "order": 1 + "typoCount": 0 + "maxTypoCount": 0 + "score": 1.0 + "proximity": + "order": 2, + "score": 1.0 + "attribute": + "order": 3 + "attribute_ranking_order_score": 1.0 + "query_word_distance_score": 1.0 + "score": 1.0 + "title:asc": + "order": 4 + "value": "batman: the dark knight returns, part 1" + "release_date:desc": + "order": 5 + "value": 1345507200.0 + "exactness": + "order": 6 + "matchType": "exactMatch" + "score": 1.0 hit: type: object additionalProperties: true @@ -168,6 +319,15 @@ components: properties: '': $ref: '#/components/schemas/matchesPosition' + _rankingScore: + type: number + description: Only present if showRankingScore = `true`. The ranking score of that document. + _rankingScoreDetails: + type: object + description: (EXPERIMENTAL) Only present if showRankingScoreDetails = `true`. The ranking score of each ranking rule for that document. + properties: + '': + $ref: '#/components/schemas/rankingScoreDetails' attribute: type: - string @@ -176,6 +336,21 @@ components: _geoDistance: type: number description: 'Using _geoPoint({lat}, {lng}) built-in sort rule at search leads the engine to return a _geoDistance within the search results. This field represents the distance in meters of the document from the specified _geoPoint.' + facetHit: + type: object + additionalProperties: true + examples: + - value: Romance + count: 25 + description: FacetHit object represents a matched facet value for a facet search. + properties: + value: + type: string + description: The facet value being matched. + additionalProperties: true + count: + type: integer + description: The number of document containing the matched facet value. documentId: oneOf: - type: number @@ -306,6 +481,27 @@ components: - hits - processingTimeMs - query + facetSearchResponse: + type: object + additionalProperties: false + title: '' + properties: + facetHits: + type: array + description: Array of facet hits + items: + $ref: '#/components/schemas/facetHit' + facetQuery: + type: string + description: Facet query originating the response. + example: ninja + processingTimeMs: + type: integer + description: Processing time of the facet search query. + required: + - facetHits + - facetQuery + - processingTimeMs task: type: object description: | @@ -570,6 +766,12 @@ components: type: integer default: 100 nullable: false + sortFacetValuesBy: + description: Defines how facet values are sorted. By default, all facets (`*`) are sorted by name, alphanumerically in ascending order (`alpha`). `count` sorts facet values by the number of documents containing a facet value in descending order. + type: object + example: + "*": 'alpha' + "genres": 'count' filterableAttributes: type: array description: | @@ -669,6 +871,11 @@ components: description: Query string. default: '""' example: '"Back to the future"' + vector: + type: array + description: Query vector. + default: 'null' + example: '[0.8, 0.145, 0.26, 0.3]' attributesToRetrieve: type: array description: 'Array of attributes whose fields will be present in the returned documents. Defaults to the [displayedAttributes list](https://docs.meilisearch.com/reference/features/settings.html#displayed-attributes) which contains by default all attributes found in the documents.' @@ -712,10 +919,22 @@ components: type: boolean description: Defines whether an `_matchesPosition` object that contains information about the matches should be returned or not. default: false + showRankingScore: + type: boolean + description: Defines whether a `_rankingScore` number representing the relevancy score of that document should be returned or not. + default: false + showRankingScoreDetails: + type: boolean + description: (EXPERIMENTAL) Defines whether a `_rankingScoreDetails` object containing information about the score of that document for each ranking rule should be returned or not. + default: false matchingStrategy: type: string description: Defines which strategy to use to match the query terms within the documents as search results. Two different strategies are available, `last` and `all`. By default, the `last` strategy is chosen. default: 'last' + attributesToSearchOn: + type: array + description: Defines which `searchableAttributes` the query will search on. + default: '["*"]' filter: $ref: '#/components/schemas/filter' facets: @@ -762,7 +981,39 @@ components: attributesToHighlight: - overview showMatchesPosition: true + showRankingScore: true wordsMatchingStrategy: all + matchingStrategy: all + facetSearchQuery: + type: object + additionalProperties: false + properties: + facetName: + type: string + required: true + description: Query string. + example: '"genres"' + facetQuery: + type: string + description: + default: '""' + example: '"Horror"' + q: + type: string + description: 'Additional search parameter. If additional search parameters are set, the method will return facet values that both: - Match the face query - Are contained in the records matching the additional search parameters' + default: '""' + example: '"Back to the future"' + matchingStrategy: + type: string + description: 'Additional search parameter. If additional search parameters are set, the method will return facet values that both: - Match the face query - Are contained in the records matching the additional search parameters' + default: 'last' + filter: + $ref: '#/components/schemas/filter' + examples: + Example: + value: + facetName: genres + facetQuery: Romance error: title: error type: object @@ -824,6 +1075,8 @@ components: - keys.create - keys.update - keys.delete + - experimental.get + - experimental.update indexes: type: array description: 'A list of accesible indexes permitted for the key. ["*"] for all indexes. The * character can be used as a wildcard when located at the last position. e.g. "products_*"" to allow access to all indexes whose names start with "products_".' @@ -1315,11 +1568,13 @@ tags: [Learn more about documents](https://docs.meilisearch.com/learn/core_concepts/documents.html). - name: Search description: | - Meilisearch exposes 3 routes to perform searches: + Meilisearch exposes 3 routes to perform document searches: * A POST route: this is the preferred route when using API authentication, as it allows [preflight request](https://developer.mozilla.org/en-US/docs/Glossary/Preflight_request) caching and better performances. * A GET route: the usage of this route is discouraged, unless you have good reason to do otherwise (specific caching abilities for example). Other than the differences mentioned above, the two routes are strictly equivalent. * A POST multi-search route allowing to perform multiple search queries in a single HTTP request. + Meilisearch exposes 1 route to perform facet searches: + * A POST facet-search route allowing to perform a facet search query on a facet in a single HTTP request. - name: Tasks description: | The `tasks` route gives information about the progress of the [asynchronous operations](https://docs.meilisearch.com/learn/advanced/asynchronous_operations.html). @@ -1349,6 +1604,10 @@ tags: During a [dump export](https://docs.meilisearch.com/reference/api/dump.html#create-a-dump), all indexes of the current instance are exported—together with their documents and settings—and saved as a single `.dump` file. During a dump import, all indexes contained in the indicated `.dump` file are imported along with their associated documents and settings. Any existing index with the same uid as an index in the dump file will be overwritten. Dump imports are [performed at launch](https://docs.meilisearch.com/reference/features/configuration.html#import-dump) using an option. + - name: Experimental + description: | + The `experimental-features` endpoint allows enabling and disabling [experimental features](https://github.com/meilisearch/engine-team/blob/main/resources/experimental-features.md) at runtime. + Experimental features are features that are not yet covered by [Meilisearch's stability guarantee](https://github.com/meilisearch/engine-team/blob/main/resources/versioning-policy.md) and that could break between minor and patch Meilisearch versions. paths: /dumps: post: @@ -1680,6 +1939,8 @@ paths: > info > Use the reserved `_geo` object to add geo coordinates to a document. `_geo` is an object made of `lat` and `lng` field. + > + > Use the reserved `_vectors` arrays of floats to add embeddings to a document. `_vectors` is an array of floats or multiple arrays of floats in an outer array. tags: - Documents security: @@ -1734,6 +1995,8 @@ paths: > info > Use the reserved `_geo` object to add geo coordinates to a document. `_geo` is an object made of `lat` and `lng` field. + > + > Use the reserved `_vectors` arrays of floats to add embeddings to a document. `_vectors` is an array of floats or multiple arrays of floats in an outer array. tags: - Documents security: @@ -2182,6 +2445,48 @@ paths: - $ref: '#/components/parameters/Content-Type' parameters: - $ref: '#/components/parameters/indexUid' + '/indexes/{indexUid}/facet-search': + post: + operationId: indexes.documents.facet.search + summary: Facet Search + description: | + Search for facet values matching a specific query for a facet. When many values exist for a facet, users need to be able to discover non-show values they can select in order to refine their faceted search. + tags: + - Search + - Facet + security: + - apiKey: [] + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/facetSearchQuery' + responses: + '200': + description: Ok + content: + application/json: + schema: + $ref: '#/components/schemas/facetSearchResponse' + examples: + Example: + value: + facetHits: + - value: Romance + count: 25 + - value: Romantic + count: 3 + facetQuery: Rom + processingTimeMs: 0 + '401': + $ref: '#/components/responses/401' + '404': + description: Not Found + parameters: + - $ref: '#/components/parameters/Content-Type' + parameters: + - $ref: '#/components/parameters/indexUid' '/indexes/{indexUid}/settings': get: operationId: indexes.settings.get @@ -2291,6 +2596,8 @@ paths: maxTotalHits: 1000 faceting: maxValuesPerFacet: 100 + sortFacetValuesBy: + "*": "alpha" '401': $ref: '#/components/responses/401' '404': @@ -2390,6 +2697,8 @@ paths: maxTotalHits: 1000 faceting: maxValuesPerFacet: 100 + sortFacetValuesBy: + "*": "alpha" responses: '202': description: Accepted @@ -2967,7 +3276,7 @@ paths: operationId: indexes.settings.faceting.update summary: Update faceting settings description: | - Update the typo tolerance faceting of an index. + Update the faceting settings of an index. > info > If the provided index does not exist, it will be created. @@ -3430,7 +3739,7 @@ paths: attributesToHighlight: - overview showMatchesPosition: true - wordsMatchingStrategy: all + matchingStrategy: all responses: '200': description: Ok @@ -3767,7 +4076,7 @@ paths: summary: Get all tasks description: 'Get all [tasks](https://docs.meilisearch.com/learn/advanced/asynchronous_operations.html)' tags: - - tasks + - Tasks responses: '200': description: OK @@ -3780,6 +4089,8 @@ paths: type: array items: $ref: '#/components/schemas/task' + total: + $ref: '#/components/schemas/total' limit: $ref: '#/components/schemas/limit' from: @@ -3788,6 +4099,7 @@ paths: $ref: '#/components/schemas/next' required: - results + - total - limit - from - next @@ -3830,6 +4142,7 @@ paths: next: null operationId: tasks.list parameters: + - $ref: '#/components/parameters/total' - $ref: '#/components/parameters/limit' - $ref: '#/components/parameters/from' - $ref: '#/components/parameters/taskFilterUids' @@ -3849,7 +4162,7 @@ paths: summary: Delete tasks description: 'Delete finished [tasks](https://docs.meilisearch.com/lean/advanced/asynchronous_operations.html)' tags: - - tasks + - Tasks responses: '202': description: Accepted @@ -3886,7 +4199,7 @@ paths: summary: Get a task description: 'Get a [task](https://docs.meilisearch.com/learn/advanced/asynchronous_operations.html) ' tags: - - tasks + - Tasks responses: '200': description: OK @@ -3935,7 +4248,7 @@ paths: summary: Cancel tasks description: 'Cancel enqueued and/or processing [tasks](https://docs.meilisearch.com/learn/advanced/asynchronous_operations.html) ' tags: - - tasks + - Tasks responses: '202': description: Accepted @@ -4004,12 +4317,93 @@ paths: $ref: '#/components/responses/401' parameters: - $ref: '#/components/parameters/taskUid' + '/experimental-features': + get: + operationId: experimental.get + security: + - apiKey: [] + summary: (EXPERIMENTAL) Get the status of runtime experimental features + description: 'Get the status of all experimental features that can be toggled at runtime' + tags: + - Experimental + responses: + '200': + description: Ok + content: + application/json: + schema: + type: object + additionalProperties: false + properties: + vectorStore: + type: boolean + scoreDetails: + type: boolean + required: + - vectorStore + - scoreDetails + examples: + Default status of the features: + value: + vectorStore: false + scoreDetails: false + '401': + $ref: '#/components/responses/401' + patch: + operationId: experimental.update + security: + - apiKey: [] + summary: (EXPERIMENTAL) Set the status of runtime experimental features + description: 'Set the status of experimental features that can be toggled at runtime' + tags: + - Experimental + requestBody: + required: true + content: + application/json: + schema: + type: object + properties: + vectorStore: + type: boolean + scoreDetails: + type: boolean + additionalProperties: false + examples: + Example: + value: + vectorStore: true + scoreDetails: false + responses: + '200': + description: Ok + content: + application/json: + schema: + type: object + additionalProperties: false + properties: + vectorStore: + type: boolean + scoreDetails: + type: boolean + required: + - vectorStore + - scoreDetails + examples: + Updated status of the feature: + value: + vectorStore: true + scoreDetails: false + '401': + $ref: '#/components/responses/401' /metrics: get: summary: (EXPERIMENTAL) Get prometheus format metrics for observability and monitoring description: 'See [technical specification](https://github.com/meilisearch/specifications/blob/main/text/0174-metrics-api.md)' tags: - - stats + - Stats + - Experimental responses: '200': description: OK @@ -4018,37 +4412,81 @@ paths: type: string examples: sampleResponse: ' - # HELP http_requests_total HTTP requests total - # TYPE http_requests_total counter - http_requests_total{method="GET",path="/metrics"} 3 - # HELP http_response_time_seconds HTTP response times - # TYPE http_response_time_seconds histogram - http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.0005"} 0 - http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.0008"} 1 - http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.00085"} 1 - http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.0009"} 1 - http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.00095"} 1 - http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.001"} 1 - http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.00105"} 1 - http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.0011"} 1 - http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.00115"} 1 - http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.0012"} 1 - http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.0015"} 1 - http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.002"} 1 - http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.003"} 1 - http_response_time_seconds_bucket{method="GET",path="/metrics",le="1"} 2 - http_response_time_seconds_bucket{method="GET",path="/metrics",le="+Inf"} 2 - http_response_time_seconds_sum{method="GET",path="/metrics"} 0.0056515409999999995 - http_response_time_seconds_count{method="GET",path="/metrics"} 2 - # HELP meilisearch_db_size_bytes Meilisearch Db Size In Bytes + # HELP meilisearch_http_response_time_seconds HTTP response times + # TYPE meilisearch_http_response_time_seconds histogram + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.001"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.002"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.003"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.004"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.005"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.006"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.007"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.008"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.009"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.01"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.02"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.03"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.04"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.05"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.06"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.07"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.08"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.09"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.1"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.2"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.3"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.4"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.5"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.6"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.7"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.8"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.9"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="1"} 0 + meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="+Inf"} 0 + meilisearch_http_response_time_seconds_sum{method="GET",path="/metrics"} 0 + meilisearch_http_response_time_seconds_count{method="GET",path="/metrics"} 0 + # HELP meilisearch_db_size_bytes Meilisearch DB Size In Bytes # TYPE meilisearch_db_size_bytes gauge - meilisearch_db_size_bytes 155648 + meilisearch_db_size_bytes 983040 + # HELP meilisearch_http_requests_total Meilisearch HTTP requests total + # TYPE meilisearch_http_requests_total counter + meilisearch_http_requests_total{method="GET",path="/metrics"} 1 # HELP meilisearch_index_count Meilisearch Index Count # TYPE meilisearch_index_count gauge meilisearch_index_count 1 # HELP meilisearch_index_docs_count Meilisearch Index Docs Count # TYPE meilisearch_index_docs_count gauge - meilisearch_index_docs_count{index="movies"} 0' + meilisearch_index_docs_count{index="mieli"} 1 + # HELP meilisearch_nb_tasks Meilisearch Number of tasks + # TYPE meilisearch_nb_tasks gauge + meilisearch_nb_tasks{kind="indexes",value="mieli"} 6 + meilisearch_nb_tasks{kind="statuses",value="canceled"} 0 + meilisearch_nb_tasks{kind="statuses",value="enqueued"} 0 + meilisearch_nb_tasks{kind="statuses",value="failed"} 2 + meilisearch_nb_tasks{kind="statuses",value="processing"} 0 + meilisearch_nb_tasks{kind="statuses",value="succeeded"} 4 + meilisearch_nb_tasks{kind="types",value="documentAdditionOrUpdate"} 2 + meilisearch_nb_tasks{kind="types",value="documentDeletion"} 2 + meilisearch_nb_tasks{kind="types",value="documentDeletionByFilter"} 0 + meilisearch_nb_tasks{kind="types",value="dumpCreation"} 0 + meilisearch_nb_tasks{kind="types",value="indexCreation"} 2 + meilisearch_nb_tasks{kind="types",value="indexDeletion"} 0 + meilisearch_nb_tasks{kind="types",value="indexSwap"} 0 + meilisearch_nb_tasks{kind="types",value="indexUpdate"} 0 + meilisearch_nb_tasks{kind="types",value="settingsUpdate"} 0 + meilisearch_nb_tasks{kind="types",value="snapshotCreation"} 0 + meilisearch_nb_tasks{kind="types",value="taskCancelation"} 0 + meilisearch_nb_tasks{kind="types",value="taskDeletion"} 0 + # HELP meilisearch_last_update Meilisearch Last Update + # TYPE meilisearch_last_update gauge + meilisearch_last_update 1689768676 + # HELP meilisearch_is_indexing Meilisearch Is Indexing + # TYPE meilisearch_is_indexing gauge + meilisearch_is_indexing 1 + # HELP meilisearch_used_db_size_bytes Meilisearch Used DB Size In Bytes + # TYPE meilisearch_used_db_size_bytes gauge + meilisearch_used_db_size_bytes 344064' operationId: metrics.get parameters: [] security: diff --git a/text/0034-telemetry-policies.md b/text/0034-telemetry-policies.md index 01e8e8b6..bfa3ac00 100644 --- a/text/0034-telemetry-policies.md +++ b/text/0034-telemetry-policies.md @@ -40,6 +40,7 @@ The collected data is sent to [Segment](https://segment.com/). Segment is a plat | Launched | Occurs when MeiliSearch is launched the first time. | | Documents Searched POST | Aggregated event on all received requests via the `POST` - `indexes/:indexUid/search` route during one hour or until a batch size reaches `500Kb`. | | Documents Searched GET | Aggregated event on all received requests via the `GET` - `/indexes/:indexUid/search` route during one hour or until a batch size reaches `500Kb`. | +| Facet Searched POST | Aggregated event on all received requests via the `POST` - `/indexes/:indexUid/facet-search` route during one hour or until a batch size reaches `500Kb`. | | Documents Searched by Multi-Search POST | Aggregated event on all received requests via the `POST`- `/multi-search` route during one hour or until a batch size reaches `500Kb`. | | Documents Added | Aggregated event on all received requests via the `POST` - `/indexes/:indexUid/documents` route during one hour or until a batch size reaches `500Kb`. | | Documents Updated | Aggregated event on all received requests via the `PUT` - `/indexes/:indexUid/documents` route during one hour or until a batch size reaches `500Kb`. | @@ -68,6 +69,9 @@ The collected data is sent to [Segment](https://segment.com/). Segment is a plat | Stats Seen | Occurs when stats are fetched via `GET` - `/stats` or `/indexes/:indexUid/stats`. | | Health Seen | Aggregated event on all received requests via the `GET - /health` route during one hour or until a batch size reaches `500Kb`. | | Version Seen | Occurs when `GET - /version` is fetched. | +| Experimental features Seen | Occurs when `GET - /experimental-features` is fetched. | +| Experimental features Updated | Occurs when experimental features are updated via `PATCH - /experimental-features` | + ---- @@ -111,7 +115,7 @@ The collected data is sent to [Segment](https://segment.com/). Segment is a plat | `stats.database_size` | Database size. Expressed in `Bytes` | 2621440 | Every hour | | `stats.indexes_number` | Number of indexes | 2 | Every hour | | `start_since_days` | Number of days since instance was launched | 365 | Every hour | -| `user_agent` | User-agent header encountered during one or more API calls | ["Meilisearch Ruby (v2.1)", "Ruby (3.0)"] | `Documents Searched POST`, `Documents Searched GET`, `Index Created`, `Index Updated`, `Documents Added`, `Documents Updated`, `Documents Deleted`, `Settings Updated`, `Ranking Rules Updated`, `SortableAttributes Updated`, `FilterableAttributes Updated`, `SearchableAttributes Updated`, `TypoTolerance Updated`, `Pagination Updated`, `Faceting Updated`, `DistinctAttribute Updated`, `DisplayedAttributes Updated`, `StopWords Updated`, `Synonyms Updated`, `Dump Created`, `Tasks Seen`, `Stats Seen`, `Health Seen`, `Version Seen`, `Documents Searched by Multi-Search POST` | +| `user_agent` | User-agent header encountered during one or more API calls | ["Meilisearch Ruby (v2.1)", "Ruby (3.0)"] | `Documents Searched POST`, `Documents Searched GET`, `Index Created`, `Index Updated`, `Documents Added`, `Documents Updated`, `Documents Deleted`, `Settings Updated`, `Ranking Rules Updated`, `SortableAttributes Updated`, `FilterableAttributes Updated`, `SearchableAttributes Updated`, `TypoTolerance Updated`, `Pagination Updated`, `Faceting Updated`, `DistinctAttribute Updated`, `DisplayedAttributes Updated`, `StopWords Updated`, `Synonyms Updated`, `Dump Created`, `Tasks Seen`, `Stats Seen`, `Health Seen`, `Version Seen`, `Documents Searched by Multi-Search POST`, `Experimental features Seen`, `Experimental features Updated` | | `requests.99th_response_time` | Highest latency from among the fastest 99% of successful search requests | 57ms | `Documents Searched POST`, `Documents Searched GET`| | `requests.total_succeeded` | Total number of successful requests in this batch | 3456 | `Documents Searched POST`, `Documents Searched GET`, `Documents Searched by Multi-Search POST` | | `requests.total_failed` | Total number of failed requests in this batch | 24 | `Documents Searched POST`, `Documents Searched GET`, `Documents Searched by Multi-Search POST` | @@ -121,7 +125,9 @@ The collected data is sent to [Segment](https://segment.com/). Segment is a plat | `filter.with_geoRadius` | `true` if the filter rule `_geoRadius` was used in this batch, otherwise `false` | false | `Documents Searched POST`, `Documents Searched GET` | | `filter.with_geoBoundingBox` | `true` if the filter rule `_geoBoundingBox` was used in this batch, otherwise `false`| false | `Documents Searched POST`, `Documents Searched GET` | | `filter.most_used_syntax` | Most used filter syntax among all requests containing the `filter` parameter in this batch | string | `Documents Searched POST`, `Documents Searched GET` | +| `attributes_to_search_on.total_number_of_uses`| Total number of queries where `attributesToSearchOn` is set | 5 | `Documents Searched POST`, `Documents Searched GET` | | `q.max_terms_number` | Highest number of terms given for the `q` parameter in this batch | 5 | `Documents Searched POST`, `Documents Searched GET` | +| `vector.max_vector_size` | Highest number of dimensions given for the `vector` parameter in this batch | 1536 | `Documents Searched POST`, `Documents Searched GET`, `Documents Searched by Multi-Search POST` | | `pagination.max_limit` | Highest value given for the `limit` parameter in this batch | 60 | `Documents Searched POST`, `Documents Searched GET`, `Documents Fetched GET`, `Documents Fetched POST` | | `pagination.max_offset` | Highest value given for the `offset` parameter in this batch | 1000 | `Documents Searched POST`, `Documents Searched GET`, `Documents Fetched GET`, `Documents Fetched POST` | | `pagination.most_used_navigation` | Most used search results navigation among all search requests in this batch. `estimated` / `exhaustive` | `estimated` | `Documents Searched POST`, `Documents Searched GET` | @@ -157,6 +163,8 @@ The collected data is sent to [Segment](https://segment.com/). Segment is a plat | `typo_tolerance.min_word_size_for_typos.two_typos`| The defined value for `minWordSizeForTypos.twoTypos` property | `9` | `Settings Updated`, `TypoTolerance Updated` | | `pagination.max_total_hits` | The defined value for `pagination.maxTotalHits` property | `1000` | `Settings Updated`, `Pagination Updated` | | `faceting.max_values_per_facet` | The defined value for `faceting.maxValuesPerFacet` property | `100` | `Settings Updated`, `Faceting Updated` | +| `faceting.sort_facet_values_by_star_count` | Whether the user set all fields to be sort by count | `true` | `Settings Updated`, `Faceting Updated` | +| `faceting.sort_facet_values_by_total` | The number of different values that were set | `10` | `Settings Updated`, `Faceting Updated` | | `distinct_attribute.set` | `true` if a field name is specified as a distrinct attribute, otherwise `false`. | `false` | `Settings Updated`, `DistinctAttribute Updated` | | `displayed_attributes.total` | Number of displayed attributes. | `3` | `SettingUpdated`, `DisplayedAttributes Updated` | | `displayed_attributes.with_wildcard` | `true` if `*` is specified as a displayed attribute, otherwise `false`. | `false` | `SettingUpdated`, `DisplayedAttributes Updated` | @@ -186,6 +194,13 @@ The collected data is sent to [Segment](https://segment.com/). Segment is a plat | `per_batch` | `true` if `POST /indexes/:indexUid/documents/delete-batch` endpoint was used in this batch, otherwise `false` | false | `Documents Deleted` | | `per_filter`| `true` if `POST /indexes/:indexUid/documents/delete` endpoint was used in this batch, otherwise `false` | false | `Documents Fetched GET`, `Documents Fetched POST`, `Documents Deleted` | | `clear_all` | `true` if `DELETE /indexes/:indexUid/documents` endpoint was used in this batch, otherwise `false` | false | `Documents Deleted` | +| vector_store | Whether the [vector store](./0193-experimental-features.md#vector-store) feature is enabled. | `true` | `Experimental features Updated` | +| score_details | Whether the [score details](./0193-experimental-features.md#score-details) feature is enabled. | `true` | `Experimental features Updated` | +| scoring.show_ranking_score | Was `showRankingScore` used in the aggregated event? If yes, `true`, otherwise `false` | `false` | `Documents Searched POST`, `Documents Searched GET`, `Documents Searched by Multi-Search POST` | +| scoring.show_ranking_score_details | Was `showRankingScoreDetails` used in the aggregated event? If yes, `true`, otherwise `false` | `false` | `Documents Searched POST`, `Documents Searched GET`, `Documents Searched GET` | +| `facets.total_distinct_facet_count` | The total number of distinct facets queried for the aggregated event | `3` | `Facet Searched POST` | +| `facets.additional_search_parameters_provided` | Were additional search parameters provided for the aggregated event | `true` | `Facet Searched POST` | + ---- @@ -267,7 +282,9 @@ This property allows us to gather essential information to better understand on | filter.with_geoBoundingBox | Does the built-in filter rule _geoBoundingBox has been used in the aggregated event?| `false` | | filter.avg_criteria_number | The average number of filter criteria among all the requests containing the `filter` parameter in the aggregated event. `"filter": []` equals to `0` while not sending `filter` does not influence the average in the aggregated event. | `4` | | filter.most_used_syntax | The most used filter syntax among all the requests containing the requests containing the `filter` parameter in the aggregated event. `string` / `array` / `mixed` | `mixed` | +| attributes_to_search_on.total_number_of_uses| Total number of queries where `attributesToSearchOn` is set | `5` | | q.max_terms_number | The maximum number of terms for the `q` parameter among all requests in the aggregated event. | `5` | +| vector.max_vector_size | The maximum number of dimensions for the `vector` parameter among all requests in the aggregated event. | `1536` | | pagination.max_limit | The maximum limit encountered among all requests in the aggregated event. | `20` | | pagination.max_offset | The maximum offset encountered among all requests in the aggregated event. | `1000` | | pagination.most_used_navigation | Most used search results navigation among all requests in the aggregated event. `estimated` / `exhaustive` | `estimated` || @@ -278,9 +295,11 @@ This property allows us to gather essential information to better understand on | formatting.max_attributes_to_crop | The maximum number of attributes to crop encountered among all requests in the aggregated event. | `100` | | formatting.crop_length | Does `cropLength` has been used in the aggregated event? If yes, `true` otherwise `false` | `false` | | formatting.crop_marker | Does `cropMarker` has been used in the aggregated event? If yes, `true` otherwise `false` | `false` | -| formatting.show_matches_position | Does `showMatchesPosition` has been used in the aggregated event? If yes, `true` otherwise `false` | `false` | +| formatting.show_matches_position | Was `showMatchesPosition` used in the aggregated event? If yes, `true` otherwise `false` | `false` | | facets.avg_facets_number | The average number of facets among all the requests containing the `facets` parameter in the aggregated event. `"facets": []` equals to `0` while not sending `facets` does not influence the average in the aggregated event. | `10` | | matching_strategy.most_used_strategy | Most used word matching strategy among all search requests in the aggregated event. `last` / `all` | `last` | +| scoring.show_ranking_score | Was `showRankingScore` used in the aggregated event? If yes, `true`, otherwise `false` | `false` | +| scoring.show_ranking_score_details | Was `showRankingScoreDetails` used in the aggregated event? If yes, `true`, otherwise `false` | `false` | --- @@ -301,6 +320,7 @@ This property allows us to gather essential information to better understand on | filter.with_geoBoundingBox | Does the built-in filter rule _geoBoundingBox has been used in the aggregated event?| `false` | | filter.avg_criteria_number | The average number of filter criteria among all the requests containing the `filter` parameter in the aggregated event. `"filter": []` equals to `0` while not sending `filter` does not influence the average in the aggregated event. | `4` | | filter.most_used_syntax | The most used filter syntax among all the requests containing the requests containing the `filter` parameter in the aggregated event. `string` / `array` / `mixed` | `mixed` | +| attributes_to_search_on.total_number_of_uses | Total number of queries where `attributesToSearchOn` is set | 5 | | q.max_terms_number | The maximum number of terms for the `q` parameter among all requests in the aggregated event. | `5` | | pagination.max_limit | The maximum limit encountered among all requests in the aggregated event. | `20` | | pagination.max_offset | The maximum offset encountered among all requests in the aggregated event. | `1000` | @@ -315,6 +335,23 @@ This property allows us to gather essential information to better understand on | formatting.show_matches_position | Does `showMatchesPosition` has been used in the aggregated event? If yes, `true` otherwise `false` | `false` | | facets.avg_facets_number | The average number of facets among all the requests containing the `facets` parameter in the aggregated event. `"facets": []` equals to `0` while not sending `facets` does not influence the average in the aggregated event. | `10` | | matching_strategy.most_used_strategy | Most used word matching strategy among all search requests in the aggregated event. `last` / `all` | `last` | +| scoring.show_ranking_score | Was `showRankingScore` used in the aggregated event? If yes, `true`, otherwise `false` | `false` | +| scoring.show_ranking_score_details | Was `showRankingScoreDetails` used in the aggregated event? If yes, `true`, otherwise `false` | `false` | + +--- +#### `Facet Searched POST` + +> The Facet Searched event is sent once an hour or when a batch reaches the maximum size of `500kb`. The event's properties are averaged over all requests on `POST` - `/indexes/:indexUid/facet-search`. + +| Property name | Description | Example | +|---------------|-------------|---------| +| user_agent | Represents all the user-agents encountered on this endpoint in the aggregated event.| `["Meilisearch Ruby (v2.1)", "Ruby (3.0)"]` | +| requests.99th_response_time | Highest latency from among the fastest 99% of successful search requests. | `57ms` | +| requests.total_succeeded | The total number of succeeded search requests in the aggregated event. | `3456` | +| requests.total_failed | The total number of failed search requests in the aggregated event. | `24` | +| requests.total_received | The total number of received search requests in the aggregated event. | `3480` | +| facets.total_distinct_facet_count | The total number of distinct facets queried for the aggregated event. | `3` | +| facets.additional_search_parameters_provided | Were additional search parameters provided for the aggregated event. | `true` | --- @@ -449,6 +486,8 @@ This property allows us to gather essential information to better understand on | typo_tolerance.min_word_size_for_typos.two_typos | The defined value for `minWordSizeForTypos.twoTypos` property. | `9` | | pagination.max_total_hits | The defined value for `pagination.maxTotalHits` property | `1000` | | faceting.max_values_per_facet | The defined value for `faceting.maxValuesPerFacet` property | `100` | +| faceting.sort_facet_values_by_star_count | Whether the user set all fields to be sort by count | `true` | +| faceting.sort_facet_values_by_total | The number of different values that were set | `10` | | distinct_attribute.set | `true` if a field name is specified, otherwise `false`. | `false` | | displayed_attributes.total | Number of displayed attributes. | `3` | | displayed_attributes.with_wildcard | `true` if `*` is specified as a displayed attribute, otherwise `false`. | `false` | @@ -522,6 +561,8 @@ This property allows us to gather essential information to better understand on |---------------|-------------|---------| | user_agent | Represents the user-agent encountered on this call. | `["Meilisearch Ruby (v2.1)", "Ruby (3.0)"]` | | faceting.max_values_per_facet | The defined value for `maxValuesPerFacet` property | `100` | +| faceting.sort_facet_values_by_star_count | Whether the user set all fields to be sort by count | `true` | +| faceting.sort_facet_values_by_total | The number of different values that were set | `10` | ## `DistinctAttribute Updated` @@ -635,6 +676,21 @@ This property allows us to gather essential information to better understand on |---------------|-------------|---------| | user_agent | Represents the user-agent encountered on this call. | `["Meilisearch Ruby (v2.1)", "Ruby (3.0)"]` | +## `Experimental features Seen` + +| Property name | Description | Example | +|---------------|-------------|---------| +| user_agent | Represents the user-agent encountered on this call. | `["Meilisearch Ruby (v2.1)", "Ruby (3.0)"]` | + +## `Experimental features Updated` + +| Property name | Description | Example | +|---------------|-------------|---------| +| user_agent | Represents the user-agent encountered on this call. | `["Meilisearch Ruby (v2.1)", "Ruby (3.0)"]` | +| vector_store | Whether the [vector store](./0193-experimental-features.md#vector-store) feature is enabled. | `true` | +| score_details | Whether the [score details](./0193-experimental-features.md#score-details) feature is enabled. | `true` | + + --- #### User-interface diff --git a/text/0060-tasks-api.md b/text/0060-tasks-api.md index cb1b3b7e..02a7e16c 100644 --- a/text/0060-tasks-api.md +++ b/text/0060-tasks-api.md @@ -564,6 +564,7 @@ The main drawback of this type of pagination is that it does not navigate within | field | type | description | |-------|------|--------------------------------------| +| total | integer | The total number of tasks matching the filter/query | | limit | integer | Default `20`. | | from | integer | The first task uid returned | | next | integer - nullable | Represents the value to send in `from` to fetch the next slice of the results. The first item for the next slice starts at this exact number. When the returned value is null, it means that all the data have been browsed in the given order. | @@ -602,6 +603,7 @@ This part demonstrates keyset paging in action on `/tasks`. The items `uid` rema ..., } ], + "total": 1351, "limit": 20, "from": 1350, "next": 1329 @@ -629,6 +631,7 @@ This part demonstrates keyset paging in action on `/tasks`. The items `uid` rema ..., } ], + "total": 1330, "limit": 50, "from": 1329, "next": 1278 @@ -656,6 +659,7 @@ This part demonstrates keyset paging in action on `/tasks`. The items `uid` rema ..., } ], + "total": 20, "limit": 20, "from": 20, "next": null diff --git a/text/0061-error-format-and-definitions.md b/text/0061-error-format-and-definitions.md index 2d1abf03..8be26abd 100644 --- a/text/0061-error-format-and-definitions.md +++ b/text/0061-error-format-and-definitions.md @@ -69,7 +69,7 @@ Errors can be returned in two different ways: `Synchronous` or `Asynchronous`. This error code is generic. Whenever an error is thrown for a resource field, a clear and precise error code should be determined to guide the user efficiently. -E.g. Sending an unknow field for a resource raises a generic `bad_request` error. +E.g. Sending an unknown field for a resource raises a generic `bad_request` error. ### Error Definition @@ -1474,6 +1474,31 @@ HTTP Code: `400 Bad Request` --- +## invalid_search_vector + +`Synchronous` + +### Context + +This error occurs for the listed reasons: +- if a value with a different type than `Array of Float` or `null` for `vector` is specified. +- if the vector length differs from the documents `_vectors` length. + +### Error Definition + +HTTP Code: `400 Bad Request` + +```json +{ + "message": "`:deserr_helper`", + "code": "invalid_search_vector", + "type": "invalid_request", + "link": "https://docs.meilisearch.com/errors#invalid_search_vector" +} +``` + +--- + ## invalid_search_offset `Synchronous` @@ -1808,6 +1833,123 @@ HTTP Code: `400 Bad Request` } ``` +## invalid_search_attributes_to_search_on + +`Synchronous` + +### Context + +This error occurs if a value with a different type than `Array of String`(POST), `String`(GET) or `null` and other than attributes names contained in the settings `searchableAttributes` as a value for `attributesToSearchOn` is specified. + +### Error Definition + +HTTP Code: `400 Bad Request` + +```json +{ + "message": "`:deserr_helper`", + "code": "invalid_search_attributes_to_search_on", + "type": "invalid_request", + "link": "https://docs.meilisearch.com/errors#invalid_search_attributes_to_search_on" +} +``` + +#### Variant: one of the values is not part of the settings `searchableAttributes` list + +```json +{ + "message": "Attribute `:value` is not searchable. Available searchable attributes are: `:searchableAttributes`.", + "code": "invalid_search_attributes_to_search_on", + "type": "invalid_request", + "link": "https://docs.meilisearch.com/errors#invalid_search_attributes_to_search_on" +} +``` + +#### Variant: one of the values is not part of the settings `searchableAttributes` list and not all of the `searchableAttributes` are displayable + +```json +{ + "message": "Attribute `:value` is not searchable. Available searchable attributes are: `:DisplayableSearchableAttributes, <..hidden-attributes>`.", + "code": "invalid_search_attributes_to_search_on", + "type": "invalid_request", + "link": "https://docs.meilisearch.com/errors#invalid_search_attributes_to_search_on" +} +``` + +--- + +## missing_facet_search_facet_name + +`Synchronous` + +### Context + +This error occurs if `facetName` isn't specified when making a facet search call. + +### Error Definition + +HTTP Code: `400 Bad Request` + +```json +{ + "message": "`:deserr_helper`", + "code": "missing_facet_search_facet_name", + "type": "invalid_request", + "link": "https://docs.meilisearch.com/errors#missing_facet_search_facet_name" +} +``` + +--- + +## invalid_facet_search_facet_name + +`Synchronous` + +### Context + +This errors occurs when the provided value for `facetName`: + +- Is not a string +- Is not defined in the `filterableAttributes` index setting + +### Error Definition + +HTTP Code: `400 Bad Request` + +```json +{ + "message": "`:deserr_helper`", + "code": "invalid_facet_search_facet_name", + "type": "invalid_request", + "link": "https://docs.meilisearch.com/errors#invalid_facet_search_facet_name" +} +``` + +--- + +## invalid_facet_search_facet_query + +`Synchronous` + +### Context + +This errors occurs when the provided value for `facetQuery`: + +- Is not a string or null + +### Error Definition + +HTTP Code: `400 Bad Request` + +```json +{ + "message": "`:deserr_helper`", + "code": "invalid_facet_search_facet_query", + "type": "invalid_request", + "link": "https://docs.meilisearch.com/errors#invalid_facet_search_facet_query" +} +``` + --- ## invalid_document_geo_field @@ -1863,6 +2005,38 @@ These errors occurs when the `_geo` field of a document payload is not valid. Ei --- +## invalid_document_vectors_field + +`Asynchronous` + +### Context + +This error occurs when the `_vectors` field of a document payload is not valid either due to the type of it or the number of dimensions. + +### Error Definition + +#### Variant: `_vectors` field value type is invalid + +```json +{ + "message": "The `_vectors` field in the document with the id: `:documentId` is not an array. Was expecting an array of floats or an array of arrays of floats but instead got `:field`", + "code": "invalid_document_vectors_type", + "type": "invalid_request", + "link": "https://docs.meilisearch.com/errors#invalid_document_vectors_type" +} +``` + +#### Variant: Number of dimensions is not correct + +```json +{ + "message": "Invalid vector dimensions: expected: `:expected`, found: `:found`.", + ... +} +``` + +--- + ## payload_too_large `Synchronous` @@ -2512,6 +2686,31 @@ HTTP Code: `400 Bad Request` - `:payloadType` is e.g. `json`, `ndjson`, `csv` +## feature_not_enabled + +`Synchronous` + +### Context + +This error occurs when a request was made using an [experimental feature](./0193-experimental-features.md) that wasn't enabled. + +### Error definition + +HTTP Code: `400 Bad Request` + +```json +{ + "message": "{:action} requires enabling the `{:featureName}` experimental feature. See {:productDiscussion}", + "code": "feature_not_enabled", + "type": "invalid_request", + "link": "https://docs.meilisearch.com/errors#feature_not_enabled" +} +``` + +- `:action` is the action that is not permitted without enabling the feature, e.g. "Getting the metrics" +- `:featureName` is the name of the feature that needs enabling, e.g. "Prometheus Metrics" +- `:productDiscussion` is the link to the product discussion that is tracking the feature + --- # internal type diff --git a/text/0085-api-keys.md b/text/0085-api-keys.md index 146c9712..685a7611 100644 --- a/text/0085-api-keys.md +++ b/text/0085-api-keys.md @@ -335,6 +335,8 @@ Create an API key. | keys.create | Provides access to `POST` `/keys` route. | | keys.update | Provides access to `PATCH` `/keys` routes. | | keys.delete | Provides access to `DELETE` `/keys` routes. | +| experimental.get | Provides access to `GET` `/experimental-features` routes. | +| experimental.update | Provides access to `PATCH` `/experimental-features` routes. | ###### 3.2.4.4.3. Response Definition diff --git a/text/0118-search-api.md b/text/0118-search-api.md index 0b3ca2c5..d95cae6b 100644 --- a/text/0118-search-api.md +++ b/text/0118-search-api.md @@ -32,25 +32,29 @@ If a master key is used to secure a Meilisearch instance, the auth layer returns ### 3.1. Search Payload Parameters -| Field | Type | Required | -|-------------------------------------------------------|--------------------------|----------| -| [`q`](#311-q) | String | False | -| [`filter`](#312-filter) | Array of String - String | False | -| [`sort`](#313-sort) | Array of String - String | False | -| [`facets`](#314-facets) | Array of String - String | False | -| [`limit`](#315-limit) | Integer | False | -| [`offset`](#316-offset) | Integer | False | -| [`page`](#317-page) | Integer | False | -| [`hitsPerPage`](#318-hitsperpage) | Integer | False | -| [`attributesToRetrieve`](#319-attributestoretrieve) | Array of String - String | False | -| [`attributesToHighlight`](#3110-attributestohighlight)| Array of String - String | False | -| [`highlightPreTag`](#3111-highlightpretag) | String | False | -| [`highlightPostTag`](#3112-highlightposttag) | String | False | -| [`attributesToCrop`](#3113-attributestocrop) | Array of String - String | False | -| [`cropLength`](#3114-croplength) | Integer | False | -| [`cropMarker`](#3115-cropmarker) | String | False | -| [`showMatchesPosition`](#3116-showmatchesposition) | Boolean | False | -| [`matchingStrategy`](#3117-matchingStrategy) | String | False | +| Field | Type | Required | +|---------------------------------------------------------------|--------------------------|----------| +| [`q`](#311-q) | String | False | +| [`filter`](#312-filter) | Array of String - String | False | +| [`sort`](#313-sort) | Array of String - String | False | +| [`facets`](#314-facets) | Array of String - String | False | +| [`limit`](#315-limit) | Integer | False | +| [`offset`](#316-offset) | Integer | False | +| [`page`](#317-page) | Integer | False | +| [`hitsPerPage`](#318-hitsperpage) | Integer | False | +| [`attributesToRetrieve`](#319-attributestoretrieve) | Array of String - String | False | +| [`attributesToHighlight`](#3110-attributestohighlight) | Array of String - String | False | +| [`highlightPreTag`](#3111-highlightpretag) | String | False | +| [`highlightPostTag`](#3112-highlightposttag) | String | False | +| [`attributesToCrop`](#3113-attributestocrop) | Array of String - String | False | +| [`cropLength`](#3114-croplength) | Integer | False | +| [`cropMarker`](#3115-cropmarker) | String | False | +| [`showMatchesPosition`](#3116-showmatchesposition) | Boolean | False | +| [`showRankingScore`](#3117-showrankingscore) | Boolean | False | +| [`showRankingScoreDetails`](#3118-showrankingscoredetails) | Boolean | False | +| [`matchingStrategy`](#3119-matchingStrategy) | String | False | +| [`attributesToSearchOn`](#3120-attributesToSearchOn) | Array of String - String | False | +| [`vector`](#3121-vector) `EXPERIMENTAL` | Array of Float | False | #### 3.1.1. `q` @@ -533,7 +537,7 @@ The first page has a value of `1`, the second `2`, etc... When `0` is provided a When providing `page` or `hitsPerPage` in the query parameters, the `page selection` system is enabled, which makes it possible to navigate through the search results pages. See explanation on the [`page selection`](#3181-navigating-search-results-by-page-selection). -If in addition to either `page` and/or `hitsPerPage`, `limit` and/or `offset` are provided as well, `limit` and `offset` are ignored. See [explaination](#3181-navigating-search-results-by-page-selection). +If in addition to either `page` and/or `hitsPerPage`, `limit` and/or `offset` are provided as well, `limit` and `offset` are ignored. See [explanation](#3181-navigating-search-results-by-page-selection). - 🔴 Sending a value with a different type than `Integer` for `page` returns an [invalid_search_page](0061-error-format-and-definitions.md#invalid_search_page) error. @@ -892,7 +896,32 @@ It's useful when more control is needed than offered by the built-in highlightin - 🔴 Sending a value with a different type than `Boolean` or `null` for `showMatchesPosition` returns an [invalid_search_show_matches_position](0061-error-format-and-definitions.md#invalid_search_show_matches_position) error. -#### 3.1.17. `matchingStrategy` +#### 3.1.17. `showRankingScore` + +- Type: Boolean +- Required: False +- Default: `false` + +Adds a [`_rankingScore`](#32114-rankingscore) number to each document in the search response, representing the relevancy score of a document according to the applied ranking rules and relative to a search query. Higher is better. + +`1.0` indicates a perfect match, `0.0` no match at all (Meilisearch should not return documents that don't match the query). + +- 🔴 Sending a value with a different type than `Boolean` or `null` for `showRankingScore` returns an [invalid_search_ranking_score](0061-error-format-and-definitions.md#invalid_search_show_ranking_score) error. + +#### 3.1.18. `showRankingScoreDetails` + +(EXPERIMENTAL) + +- Type: Object +- Required: False +- Default: `false` + +Adds a [`_rankingScoreDetails`](#32115-rankingscoredetails) object to each document in the search response, containing information about the score of that document for each applied ranking rule. + +- 🔴 Sending a value with a different type than `Boolean` or `null` for `showRankingScoreDetails` returns an [invalid_search_ranking_score_details](0061-error-format-and-definitions.md#invalid_search_show_ranking_score_details) error. +- 🔴 Using that field while the [`score details`](./0193-experimental-features.md#score-details) experimental feature has not been enabled returns a [feature_not_enabled](0061-error-format-and-definitions.md#feature_not_enabled) error. + +#### 3.1.19. `matchingStrategy` - Type: String - Required: False @@ -904,30 +933,55 @@ Two different strategies are available, `last` and `all`. By default, the `last` - 🔴 Sending a value with a different type than `String` and other than `last` or `all` as a value for `matchingStrategy` returns an [invalid_search_matching_strategy](0061-error-format-and-definitions.md#invalid_search_matching_strategy) error. -##### 3.1.17.1. `last` strategy +##### 3.1.19.1. `last` strategy The documents containing ALL the query words (i.e. in the `q` parameter) are returned first by Meilisearch. If Meilisearch doesn't have enough documents to fit the requested `limit`, it iteratively ignores the query words from the last typed word to the first typed word to match more documents. -##### 3.1.17.2. `all` strategy +##### 3.1.19.2. `all` strategy Only the documents containing ALL the query words (i.e. in the `q` parameter) are returned by Meilisearch. If Meilisearch doesn't have enough documents to fit the requested `limit`, it returns the documents found without trying to match more documents. +#### 3.1.20. `attributesToSearchOn` + +- Type: Array of String (POST) | String (GET) +- Required: False +- Default: `["*"]` + +Defines which `searchableAttributes` the query will search on. + +- If `attributesToSearchOn` is not set, set to `["*"]` or set to `null`, then the query will search on all `searchableAttributes`. +- Sending the attributes in a different order than the order set in the settings `searchableAttributes` doesn't reorder the fields' rank for the `Attributes` ranking rule +- 🔴 Sending a value with a different type than `Array of String`(POST), `String`(GET) or `null` for `attributesToSearchOn` returns an [invalid_attributes_to_search_on](0061-error-format-and-definitions.md#invalid_search_attributes_to_search_on) error. +- 🔴 Sending an attribute that is not part of the settings `searchableAttributes` list returns an [invalid_attributes_to_search_on](0061-error-format-and-definitions.md#invalid_search_attributes_to_search_on) error. + +#### 3.1.21. `vector` `EXPERIMENTAL` + +- Type: Array of Float +- Required: False +- Default: [] + +Request the nearest documents based on the query vector embedding given. + +- 🔴 Sending a value with a different type than `Array of Float` or `null` as a value for `vector` returns an [invalid_search_vector](0061-error-format-and-definitions.md#invalid_search_vector) error. +- 🔴 Sending a value for `vector` whose length differs from the documents `_vectors` length returns an [invalid_search_vector](0061-error-format-and-definitions.md#invalid_search_vector) error. + ### 3.2. Search Response Properties -| Field | Type | Required | -|-------------------------------------------------|------------|----------| -| [`hits`](#321-hits) | Array[Hit] | True | -| [`limit`](#322-limit) | Integer | False | -| [`offset`](#323-offset) | Integer | False | -| [`estimatedTotalHits`](#324-estimatedTotalHits) | Integer | False | -| [`page`](#325-page) | Integer | False | -| [`hitsPerPage`](#326-hitsperpage) | Integer | False | -| [`totalPages`](#327-totalpages) | Integer | False | -| [`totalHits`](#328-totalhits) | Integer | False | -| [`facetDistribution`](#329-facetdistribution) | Object | False | -| [`facetStats`](#3210-facetstats) | Object | False | -| [`processingTimeMs`](#3211-processingtimems) | Integer | True | -| [`query`](#3212-query) | String | True | +| Field | Type | Required | +|-------------------------------------------------|----------------|-----------| +| [`hits`](#321-hits) | Array[Hit] | True | +| [`limit`](#322-limit) | Integer | False | +| [`offset`](#323-offset) | Integer | False | +| [`estimatedTotalHits`](#324-estimatedTotalHits) | Integer | False | +| [`page`](#325-page) | Integer | False | +| [`hitsPerPage`](#326-hitsperpage) | Integer | False | +| [`totalPages`](#327-totalpages) | Integer | False | +| [`totalHits`](#328-totalhits) | Integer | False | +| [`facetDistribution`](#329-facetdistribution) | Object | False | +| [`facetStats`](#3210-facetstats) | Object | False | +| [`processingTimeMs`](#3211-processingtimems) | Integer | True | +| [`query`](#3212-query) | String | True | +| [`vector`](#3213-vector) `EXPERIMENTAL` | Array of Float | False | #### 3.2.1. `hits` @@ -940,15 +994,18 @@ Results of the search query as an array of documents. > The search parameters `attributesToRetrieve` influence the returned payload for a hit. See [3.1.7. `attributesToRetrieve`](#319-attributestoretrieve) section. -A search result can contain special properties. See [3.2.1.1. `hit` Special Properties](#3211-hits-special-properties) section. +A search result can contain special properties. See [3.2.1.1. `hit` Special Properties](#3211-hit-special-properties) section. ##### 3.2.1.1. `hit` Special Properties -| Field | Type | Required | -|----------------------------------------------|---------|----------| -| [`_geoDistance`](#32111-geodistance) | Integer | False | -| [`_formatted`](#32112-formatted) | Object | False | -| [`_matchesPosition`](#32113-matchesposition) | Object | False | +| Field | Type | Required | +|------------------------------------------------------|---------|----------| +| [`_geoDistance`](#32111-geodistance) | Integer | False | +| [`_formatted`](#32112-formatted) | Object | False | +| [`_matchesPosition`](#32113-matchesposition) | Object | False | +| [`_rankingScore`](#32114-rankingscore) | Number | False | +| [`_rankingScoreDetails`](#32115-rankingscoredetails) | Object | False | +| [`_semanticScore`](#32116-semanticscore) `EXPERIMENTAL` | Float | False | ###### 3.2.1.1.1. `_geoDistance` @@ -1155,6 +1212,37 @@ The beginning of a matching term within a field is indicated by `start`, and its > See [3.1.14. `showMatchesPosition`](#3116-showmatchesposition) section. +###### 3.2.1.1.4. `_rankingScore` + +- Type: Number +- Required: False + +The relevancy score of a document relative to the search query. Higher is better. + +`1.0` indicates a perfect match, `0.0` no match at all (Meilisearch should not return documents that don't match the query). + +> See [Ranking Score](./0195-ranking-score.md#31-ranking-score) for details. + +###### 3.2.1.1.5. `_rankingScoreDetails` + +- Type: Object +- Required: False + +(EXPERIMENTAL) The ranking score of a document per each ranking rule and relative to the search query. + +This object features one field for each applied ranking rule, whose values are an object with at least the field `order` indicating in which order this ranking rule has been applied. + +> See [Ranking Score details](./0195-ranking-score.md#32-ranking-score-details) for details. + +###### 3.2.1.1.6. `_semanticScore` `EXPERIMENTAL` + +- Type: Float +- Required: False + +Contains the semantic similarity score of the document for a vector search when `vector` has been provided. The score is represented as a dot product. + +> See [3.1.18 `vector`](#3118-vector-experimental) + #### 3.2.2. `limit` - Type: Integer @@ -1271,6 +1359,15 @@ Query originating the response. Equals to the `q` search parameter. > See [3.1.1. `q`](#311-q) section. +#### 3.2.13. `vector` `EXPERIMENTAL` + +- Type: Array of Float +- Required: False + +Vector query embedding originating the response. Equals to the `vector` search parameter if specified. + +> See [3.1.18. `vector`](#3118-vector-experimental) + ## 2. Technical Details n/a diff --git a/text/0124-documents-api.md b/text/0124-documents-api.md index 6bab2579..1f9e70d6 100644 --- a/text/0124-documents-api.md +++ b/text/0124-documents-api.md @@ -261,7 +261,31 @@ A document represented as a JSON object. - 🔴 If the requested `document_id` does not exist, the API returns an [document_not_found](0061-error-format-and-definitions.md#document_not_found) error. - 🔴 Sending a value with a different type than `String` or `null` for `fields` will return a [invalid_document_fields](0061-error-format-and-definitions.md#invalid_document_fields) error. -#### 3.1.3. `POST` - `indexes/:index_uid/documents` +#### 3.1.3. Documents Body Special Properties + +While the body of a document can contain any pair of keys and values, Meilisearch uses specific key names to leverage some search capabilities such as geo search and vector search. + +| Field | Type | Required | +|----------------------------------------------------------|--------------------------|----------| +| [`_geo`](#3131-_geo) | String | Object | False | +| [`_vectors`](#3132-_vectors-experimental) `EXPERIMENTAL` | Array of Float | Array[Array of Float] | False | + +##### 3.1.3.1. `_geo` + +Holds latitude and longitude geo coordinates for a document. + +Refer to the [geo search specification](0059-geo-search.md) + +##### 3.1.3.2. `_vectors` `EXPERIMENTAL` + +Type: Array of Float | Array[Array of Float] +Required: False + +Holds a vectorized representation of a document. It is possible to send either one or several vectorized representations of the same document. + +- 🔴 Sending a value with a different type than `Array of Float`, `Array[Array of Float]` or `null` as a value for `_vectors` returns an [invalid_document_vectors_field](0061-error-format-and-definitions.md#invalid_document_vectors_field) error. +- 🔴 Sending a value for `_vectors` whose length differs from another document `_vectors` field returns an [invalid_document_vectors_field](0061-error-format-and-definitions.md#invalid_document_vectors_field) error. +#### 3.1.4. `POST` - `indexes/:index_uid/documents` Add a list of documents or replace them if they already exist. @@ -277,26 +301,26 @@ This endpoint accepts various content-type: - [`text/csv`](0028-indexing-csv.md) - [`application/x-ndjson`](0029-indexing-ndjson.md) -##### 3.1.3.1. Path Parameters +##### 3.1.4.1. Path Parameters | Field | Type | Required | |--------------------------|--------------------------|----------| | `index_uid` | String | True | -###### 3.1.3.1.1 `index_uid` +###### 3.1.4.1.1 `index_uid` - Type: String - Required: True Unique identifier of an index. -##### 3.1.3.2. Request Payload Definition +##### 3.1.4.2. Request Payload Definition | Field | Type | Required | |--------------------------|--------------------------|----------| | `primaryKey` | String | False | -###### 3.1.3.2.1 `primaryKey` +###### 3.1.4.2.1 `primaryKey` - Type: String - Required: False @@ -312,13 +336,13 @@ When the index is empty, it is possible to modify the `primaryKey`. If the index is not empty, the query parameter `primaryKey` is ignored. -##### 3.1.3.3. Response Definition +##### 3.1.4.3. Response Definition When the request is successful, Meilisearch returns the HTTP code `202 Accepted`. The response's content is the summarized representation of the received asynchronous task. See [Summarized `task` Object for `202 Accepted`](0060-tasks-api.md#summarized-task-object-for-202-accepted). -##### 3.1.3.4. Errors +##### 3.1.4.4. Errors - 🔴 Omitting Content-Type header returns a [missing_content_type](0061-error-format-and-definitions.md#missing_content_type) error. - 🔴 Sending an empty Content-Type returns an [invalid_content_type](0061-error-format-and-definitions.md#invalid_content_type) error. @@ -328,16 +352,16 @@ See [Summarized `task` Object for `202 Accepted`](0060-tasks-api.md#summarized-t - 🔴 Sending an invalid index uid format for the `:index_uid` path parameter returns an [invalid_index_uid](0061-error-format-and-definitions.md#invalid_index_uid) error. - 🔴 Sending a value with a different type than `String` or `null` for the `primaryKey` parameter will return a [invalid_index_primary_key](0061-error-format-and-definitions.md#invalid_index_primary_key) error. -###### 3.1.3.4.1. Async Errors +###### 3.1.4.4.1. Async Errors - 🔴 When Meilisearch is secured, if the API Key do not have the `indexes.create` action defined, the API returns an [index_not_found](0061-error-format-and-definitions.md#index_not_found) error in the related asynchronous `task` resource. See [3.2.2.2. Response Definition](#3222-response-definition). - 🔴 When updating the `primaryKey`, if the previous `primaryKey` value has already been used for a document, the API returns an [index_primary_key_already_exists](0061-error-format-and-definitions.md#index_primary_key_already_exists) error. -##### 3.1.3.5. Lazy Index Creation +##### 3.1.4.5. Lazy Index Creation If the requested `index_uid` does not exist, and the authorization layer allows it (See [3.1.3.4.1. Async Errors](#31341-async-errors)), Meilisearch will create the index when the related asynchronous task resource is executed. See [3.1.3.3. Response Definition](#3133-response-definition). -#### 3.1.4. `PUT` - `indexes/:index_uid/documents` +#### 3.1.5. `PUT` - `indexes/:index_uid/documents` Add a list of documents or update them if they already exist. @@ -351,26 +375,26 @@ This endpoint accepts various content-type: - [`text/csv`](0028-indexing-csv.md) - [`application/x-ndjson`](0029-indexing-ndjson.md) -##### 3.1.4.1. Path Parameters +##### 3.1.5.1. Path Parameters | Field | Type | Required | |--------------------------|--------------------------|----------| | `index_uid` | String | True | -###### 3.1.4.1.1. `index_uid` +###### 3.1.5.1.1. `index_uid` - Type: String - Required: True Unique identifier of an index. -##### 3.1.4.2. Query Parameters Definition +##### 3.1.5.2. Query Parameters Definition | Field | Type | Required | |--------------------------|--------------------------|----------| | `primaryKey` | String | False | -###### 3.1.4.2.1. `primaryKey` +###### 3.1.5.2.1. `primaryKey` - Type: String - Required: False @@ -386,13 +410,13 @@ When the index is empty, it is possible to modify the `primaryKey`. If the index is not empty, the query parameter `primaryKey` is ignored. -##### 3.1.4.3. Response Definition +##### 3.1.5.3. Response Definition When the request is successful, Meilisearch returns the HTTP code `202 Accepted`. The response's content is the summarized representation of the received asynchronous task. See [Summarized `task` Object for `202 Accepted`](0060-tasks-api.md#summarized-task-object-for-202-accepted). -##### 3.1.4.4. Errors +##### 3.1.5.4. Errors - 🔴 Omitting Content-Type header returns a [missing_content_type](0061-error-format-and-definitions.md#missing_content_type) error. - 🔴 Sending an empty Content-Type returns an [invalid_content_type](0061-error-format-and-definitions.md#invalid_content_type) error. @@ -402,109 +426,109 @@ See [Summarized `task` Object for `202 Accepted`](0060-tasks-api.md#summarized-t - 🔴 Sending an invalid index uid format for the `:index_uid` path parameter returns an [invalid_index_uid](0061-error-format-and-definitions.md#invalid_index_uid) error. - 🔴 Sending a value with a different type than `String` or `null` for the `primaryKey` parameter will return a [invalid_index_primary_key](0061-error-format-and-definitions.md#invalid_index_primary_key) error. -###### 3.1.4.4.1. Async Errors +###### 3.1.5.4.1. Async Errors - 🔴 When Meilisearch is secured, if the API Key do not have the `indexes.create` action defined, the API returns an [index_not_found](0061-error-format-and-definitions.md#index_not_found) error in the related asynchronous `task` resource. See [3.2.2.2. Response Definition](#3222-response-definition). - 🔴 When updating the `primaryKey`, if the previous `primaryKey` value has already been used for a document, the API returns an [index_primary_key_already_exists](0061-error-format-and-definitions.md#index_primary_key_already_exists) error. -##### 3.1.4.5. Lazy Index Creation +##### 3.1.5.5. Lazy Index Creation If the requested `index_uid` does not exist, and the authorization layer allows it (See [3.1.4.4.1. Async Errors](#31441-async-errors)), Meilisearch will create the index when the related asynchronous task resource is executed. See [3.1.4.3. Response Definition](#3143-response-definition). -#### 3.1.5. `DELETE` - `indexes/:index_uid/documents` +#### 3.1.6. `DELETE` - `indexes/:index_uid/documents` Delete all documents in the specified index. -##### 3.1.5.1. Path Parameters +##### 3.1.6.1. Path Parameters | Field | Type | Required | |--------------------------|--------------------------|----------| | `index_uid` | String | true | -###### 3.1.5.1.1. `index_uid` +###### 3.1.6.1.1. `index_uid` - Type: String - Required: True Unique identifier of an index. -##### 3.1.5.2. Request Payload Definition +##### 3.1.6.2. Request Payload Definition N/A -##### 3.1.5.3. Response Definition +##### 3.1.6.3. Response Definition When the request is successful, Meilisearch returns the HTTP code `202 Accepted`. The response's content is the summarized representation of the received asynchronous task. See [Summarized `task` Object for `202 Accepted`](0060-tasks-api.md#summarized-task-object-for-202-accepted). -##### 3.1.5.4. Errors +##### 3.1.6.4. Errors - 🔴 Sending an invalid index uid format for the `:index_uid` path parameter returns an [invalid_index_uid](0061-error-format-and-definitions.md#invalid_index_uid) error. -###### 3.1.5.4.1. Async Errors +###### 3.1.6.4.1. Async Errors - 🔴 If the requested `index_uid` does not exist, the API returns an [index_not_found](0061-error-format-and-definitions.md#index_not_found) error. -#### 3.1.6. `DELETE` - `indexes/:index_uid/documents/:document_id` +#### 3.1.7. `DELETE` - `indexes/:index_uid/documents/:document_id` Delete one document based on its unique id. -##### 3.1.6.1. Path Parameters +##### 3.1.7.1. Path Parameters | Field | Type | Required | |--------------------------|--------------------------|----------| | `index_uid` | String | True | | `document_id` | String | True | -###### 3.1.6.1.1. `index_uid` +###### 3.1.7.1.1. `index_uid` - Type: String - Required: True Unique identifier of an index. -###### 3.1.6.1.2. `document_id` +###### 3.1.7.1.2. `document_id` - Type: Integer - Required: True Unique identifier of a document. -##### 3.1.6.2. Request Payload Definition +##### 3.1.7.2. Request Payload Definition N/A -##### 3.1.6.3. Response Definition +##### 3.1.7.3. Response Definition When the request is successful, Meilisearch returns the HTTP code `202 Accepted`. The response's content is the summarized representation of the received asynchronous task. See [Summarized `task` Object for `202 Accepted`](0060-tasks-api.md#summarized-task-object-for-202-accepted). -##### 3.1.6.4. Errors +##### 3.1.7.4. Errors - 🔴 Sending an invalid index uid format for the `:index_uid` path parameter returns an [invalid_index_uid](0061-error-format-and-definitions.md#invalid_index_uid) error. -###### 3.1.6.4.1. Async Errors +###### 3.1.7.4.1. Async Errors - 🔴 If the requested `index_uid` does not exist, the API returns an [index_not_found](0061-error-format-and-definitions.md#index_not_found) error. -#### 3.1.7. `POST` - `indexes/:index_uid/documents/delete-batch` +#### 3.1.8. `POST` - `indexes/:index_uid/documents/delete-batch` Delete a selection of documents based on array of document id's. -##### 3.1.7.1. Path Parameters +##### 3.1.8.1. Path Parameters | Field | Type | Required | |--------------------------|--------------------------|----------| | `index_uid` | String | True | -###### 3.1.7.1.1. `index_uid` +###### 3.1.8.1.1. `index_uid` - Type: String - Required: True Unique identifier of an index. -##### 3.1.7.2 Request Payload Definition +##### 3.1.8.2 Request Payload Definition An array of document ids to delete. @@ -517,13 +541,13 @@ e.g. [122, 1194, 2501] ``` -##### 3.1.7.3. Response Definition +##### 3.1.8.3. Response Definition When the request is successful, Meilisearch returns the HTTP code `202 Accepted`. The response's content is the summarized representation of the received asynchronous task with the type `documentDeletion`. See [Summarized `task` Object for `202 Accepted`](0060-tasks-api.md#summarized-task-object-for-202-accepted). -##### 3.1.7.4. Errors +##### 3.1.8.4. Errors - 🔴 Omitting Content-Type header returns a [missing_content_type](0061-error-format-and-definitions.md#missing_content_type) error. - 🔴 Sending an empty Content-Type returns an [invalid_content_type](0061-error-format-and-definitions.md#invalid_content_type) error. @@ -533,28 +557,28 @@ See [Summarized `task` Object for `202 Accepted`](0060-tasks-api.md#summarized-t - 🔴 Sending an invalid index uid format for the `:index_uid` path parameter returns an [invalid_index_uid](0061-error-format-and-definitions.md#invalid_index_uid) error. - 🔴 Sending a value with a different type than `Array` for the body request will return a [bad_request](0061-error-format-and-definitions.md#bad_request) error. -###### 3.1.7.4.1 Async Errors +###### 3.1.8.4.1 Async Errors - 🔴 If the requested `index_uid` does not exist, the API returns an [index_not_found](0061-error-format-and-definitions.md#index_not_found) error. -#### 3.1.8. `POST` - `indexes/:index_uid/documents/delete` +#### 3.1.9. `POST` - `indexes/:index_uid/documents/delete` Delete a selection of documents based on a filter. -##### 3.1.8.1. Path Parameters +##### 3.1.9.1. Path Parameters | Field | Type | Required | |--------------------------|--------------------------|----------| | `index_uid` | String | True | -###### 3.1.8.1.1. `index_uid` +###### 3.1.9.1.1. `index_uid` - Type: String - Required: True Unique identifier of an index. -##### 3.1.8.2 Request Payload Definition +##### 3.1.9.2 Request Payload Definition A filter. @@ -569,13 +593,13 @@ e.g. } ``` -##### 3.1.8.3. Response Definition +##### 3.1.9.3. Response Definition When the request is successful, Meilisearch returns the HTTP code `202 Accepted`. The response's content is the summarized representation of the received asynchronous task with the type `documentDeletion`. See [Summarized `task` Object for `202 Accepted`](0060-tasks-api.md#summarized-task-object-for-202-accepted). -##### 3.1.8.4. Errors +##### 3.1.9.4. Errors - 🔴 Omitting Content-Type header returns a [missing_content_type](0061-error-format-and-definitions.md#missing_content_type) error. - 🔴 Sending an empty Content-Type returns an [invalid_content_type](0061-error-format-and-definitions.md#invalid_content_type) error. @@ -586,15 +610,15 @@ See [Summarized `task` Object for `202 Accepted`](0060-tasks-api.md#summarized-t - 🔴 Sending a value without a filter will return a [missing_document_filter](0061-error-format-and-definitions.md#missing_document_filter) error. - 🔴 Sending a value with an invalid or empty filter will return an [invalid_document_filter](0061-error-format-and-definitions.md#invalid_document_filter) error. -###### 3.1.8.4.1 Async Errors +###### 3.1.9.4.1 Async Errors - 🔴 If the requested `index_uid` does not exist, the API returns an [index_not_found](0061-error-format-and-definitions.md#index_not_found) error. -#### 3.1.9. General Errors +#### 3.1.10. General Errors These errors apply to all endpoints described here. -##### 3.1.9.1 Auth Errors +##### 3.1.10.1 Auth Errors The auth layer can return the following errors if Meilisearch is secured (a master-key is defined). @@ -606,4 +630,4 @@ N/A ## 5. Future Possibilities -- Introduce a way to reject fields from a document in the response. e.g. `?fields=-createdAt` \ No newline at end of file +- Introduce a way to reject fields from a document in the response. e.g. `?fields=-createdAt` diff --git a/text/0174-metrics-api.md b/text/0174-metrics-api.md index 8f1f7e44..a1d1c806 100644 --- a/text/0174-metrics-api.md +++ b/text/0174-metrics-api.md @@ -1,6 +1,6 @@ # Metrics API -This endpoint is currently experimental. +This endpoint is currently [experimental](./0193-experimental-features.md). This means that it can break at any time between two minor versions as long as it is not stabilized. @@ -8,13 +8,13 @@ This means that it can break at any time between two minor versions as long as i This specification describes the metrics API endpoint with the exhaustive list of returned metrics. -The endpoint returns observability data to monitor a Meilisearch instance using [prometheus](https://prometheus.io/). +The endpoint returns observability data to monitor a Meilisearch instance using [Prometheus](https://prometheus.io/). ## 2. Motivation Improve the capabilities of a Meilisearch instance regarding observability and ease its integration into monitoring stacks. -## 3. Functionnal Specification +## 3. Functional Specification ### 3.1. Activating the feature @@ -24,7 +24,7 @@ By default, the `/metrics` endpoint is not accessible. To activate it, the `--ex Prometheus metrics format is text-based and line-oriented. Lines are separated by a line feed character (n). -A metric is composed by several fields: +A metric is composed of several fields: - `# HELP` metadata - `# TYPE` metadata @@ -33,25 +33,29 @@ A metric is composed by several fields: Meilisearch returns the metrics specified in the table below. -| Name | Type | -|---------------------------------------------------------------------------|-----------| -| [`http_requests_total`](#321-http_requests_total) | counter | -| [`http_response_time_seconds`](#322-http_response_time_seconds) | histogram | -| [`meilisearch_database_size_bytes`](#323-meilisearch_database_size_bytes) | gauge | -| [`meilisearch_index_docs_count`](#324-meilisearch_index_docs_count) | gauge | -| [`meilisearch_index_count`](#325-meilisearch_index_count) | gauge | +| Name | Type | +|----------------------------------------------------------------------------------------|-----------| +| [`meilisearch_http_requests_total`](#321-meilisearch_http_requests_total) | counter | +| [`meilisearch_http_response_time_seconds`](#322-meilisearch_http_response_time_seconds)| histogram | +| [`meilisearch_db_size_bytes`](#323-meilisearch_db_size_bytes) | gauge | +| [`meilisearch_used_db_size_bytes`](#324-meilisearch_used_db_size_bytes) | gauge | +| [`meilisearch_index_docs_count`](#325-meilisearch_index_docs_count) | gauge | +| [`meilisearch_index_count`](#326-meilisearch_index_count) | gauge | +| [`meilisearch_nb_tasks`](#327-meilisearch_nb_tasks) | counter | +| [`meilisearch_last_update`](#328-meilisearch_last_update) | gauge | +| [`meilisearch_is_indexing`](#329-meilisearch_is_indexing) | gauge | -#### 3.2.1 `http_requests_total` +#### 3.2.1 `meilisearch_http_requests_total` Returns the number of times an API resource is accessed. ``` # HELP http_requests_total HTTP requests total # TYPE http_requests_total counter -http_requests_total{method=":httpMethod",path=":resourcePath"} :numberOfRequest +meilisearch_http_requests_total{method=":httpMethod",path=":resourcePath"} :numberOfRequest ``` -#### 3.2.2. `http_responses_time_seconds` +#### 3.2.2. `meilisearch_http_response_time_seconds` Returns a time histogram showing the number of times an API resource call goes into a time bucket (expressed in second). @@ -74,12 +78,13 @@ http_response_time_seconds_bucket{method=":httpMethod",path=":resourcePath",le=" http_response_time_seconds_bucket{method=":httpMethod",path=":resourcePath",le="1"} :numberOfRequest http_response_time_seconds_bucket{method=":httpMethod",path=":resourcePath",le="+Inf"} :numberOfRequest http_response_time_seconds_sum{method=":httpMethod",path=":resourcePath"} :numberOfRequest -http_response_time_seconds_count{method=":httpMethod",path=":resourcePath"} :numberOfRequest +meilisearch_http_response_time_seconds_count{method=":httpMethod",path=":resourcePath"} :numberOfRequest ``` -#### 3.2.3. `meilisearch_database_size_bytes` +#### 3.2.3. `meilisearch_db_size_bytes` -Returns the size of the database in bytes. +Returns the “real” size of the database on disk in bytes. +It includes all the lmdb memory mapped files plus all the files contained in the `data.ms` directory (mainly the updates files that were not processed yet). ``` # HELP meilisearch_db_size_bytes Meilisearch Db Size In Bytes @@ -87,7 +92,19 @@ Returns the size of the database in bytes. meilisearch_db_size_bytes :databaseSizeInBytes ``` -#### 3.2.4. `meilisearch_index_docs_count` +#### 3.2.4. `meilisearch_used_db_size_bytes` + +Returns the size of the database actually used by meilisearch in bytes. +Include all the same files as `meilisearch_db_size_bytes` except that when it comes to an LMDB database, we only count the pages used by meilisearch. +This means if you see a large gap between both metrics, adding documents will probably re-use freed pages instead of growing `meilisearch_db_size_bytes`. + +``` +# HELP meilisearch_used_db_size_bytes Meilisearch Used DB Size In Bytes +# TYPE meilisearch_used_db_size_bytes gauge +meilisearch_used_db_size_bytes :databaseSizeInBytes +``` + +#### 3.2.5. `meilisearch_index_docs_count` Returns the number of documents for an index. @@ -97,9 +114,9 @@ Returns the number of documents for an index. meilisearch_index_docs_count{index=":indexUid"} :numberOfDocuments ``` -#### 3.2.5. `meilisearch_index_count` +#### 3.2.6. `meilisearch_index_count` -Returns the total number of index for the Meilisearch instance. +Returns the total number of indexes for the Meilisearch instance. ``` # HELP meilisearch_index_count Meilisearch Index Count @@ -107,6 +124,44 @@ Returns the total number of index for the Meilisearch instance. meilisearch_index_count :numberOfIndex ```` +#### 3.2.7. `meilisearch_nb_tasks` + +Returns the total number of tasks for the Meilisearch instance parametrized by the kind of task and its value (see the table below). + +``` +# HELP meilisearch_nb_tasks Meilisearch Number of tasks +# TYPE meilisearch_nb_tasks gauge +meilisearch_nb_tasks{kind=":kind",value=":value"} :number +```` + +Here is a list of available kind and associated values: + +| Kind | values | +|-----------|------------------------------------------------------------------------| +| indexes | Any created indexes | +| statuses | Any task statuses (i.e.: succeeded, failed, enqueued, etc...) | +| types | Any task types (i.e.: documentAdditionOrUpdate, settingsUpdate, etc...)| + +#### 3.2.8. `meilisearch_last_update` + +Returns the timestamp of the last update. + +``` +# HELP meilisearch_last_update Meilisearch Last Update +# TYPE meilisearch_last_update gauge +meilisearch_last_update :unixTimestamp +```` + +#### 3.2.8. `meilisearch_is_indexing` + +Returns `1` if Meilisearch is indexing or `0` if not. + +``` +# HELP meilisearch_is_indexing Meilisearch Is Indexing +# TYPE meilisearch_is_indexing gauge +meilisearch_is_indexing :isIndexing +```` + ### 3.3. API Endpoints Definition #### 3.3.1. `GET` - `/metrics` @@ -130,7 +185,7 @@ meilisearch_total_index 2 #### 3.3.2. Errors -- 🔴 If `--experimental-enable-metrics` CLI option / `MEILI_EXPERIMENTAL_ENABLE_METRICS` env var is not specified at launch, the API returns a `404 Not Found` HTTP response. +- 🔴 If `--experimental-enable-metrics` CLI option / `MEILI_EXPERIMENTAL_ENABLE_METRICS` env var is not specified at launch, the API returns a [feature_not_enabled](./0061-error-format-and-definitions.md#feature-not-enabled) error. ##### 3.3.2.1 Auth Errors @@ -138,11 +193,11 @@ If a master key is used to secure a Meilisearch instance, the auth layer returns - 🔴 Accessing this route without the `Authorization` header returns a [missing_authorization_header](0061-error-format-and-definitions.md#missing_authorization_header) error. - 🔴 Accessing this route with a key that does not have the permission `metrics.get` (i.e. other than the master key) returns an [invalid_api_key](0061-error-format-and-definitions.md#invalid_api_key) error. -- 🔴 Accessing this route with a key that have a restriction on the indexes returns an [invalid_api_key](0061-error-format-and-definitions.md#invalid_api_key) error. +- 🔴 Accessing this route with a key that has a restriction on the indexes returns an [invalid_api_key](0061-error-format-and-definitions.md#invalid_api_key) error. ## 4. Technical Details N/A ## 5. Future Possibilities +- Merge `/stats` with `/metrics`. A header could specify the preferred format. e.g `application/json` (similar to actual `stats` resource) or `text/plain` (Prometheus) -- Merge `/stats` with `/metrics`. A header could specify the prefered format. e.g `application/json` (similar to actual `stats` resource) or `text/plain` (prometheus) \ No newline at end of file diff --git a/text/0193-experimental-features.md b/text/0193-experimental-features.md new file mode 100644 index 00000000..f3eb915e --- /dev/null +++ b/text/0193-experimental-features.md @@ -0,0 +1,98 @@ +# Experimental features + +## 1. Summary + +Experimental features are features that are not covered by [Meilisearch's stability guarantee](https://github.com/meilisearch/engine-team/blob/main/resources/versioning-policy.md). They require explicit opt-in to be usable and may break back compatibility or be removed between minor and patch versions of Meilisearch. + +## 2. Motivation + +See [Motivation](https://github.com/meilisearch/engine-team/blob/main/resources/experimental-features.md#motivation) in the experimental feature process. + +## 3. Functional Specification + +Some experimental features can be enabled when starting the Meilisearch binary via a command line option, a configuration option or an environment variable. These experimental features are the [instance experimental features](#31-instance-experimental-features). + +Other experimental features can be enabled at runtime by using the [`/experimental-features` route](./0194-experimental-feature-api.md). They are the [runtime experimental features](#32-runtime-experimental-features). + +## 3.1. Instance experimental features + +To enable instance experimental features, pass their associated command line flag as an option to Meilisearch. For each such command line flag, there also exists a configuration option and an environment variable. + +|Feature name|Command line flag|Description|Missing for stabilization|Expected stabilization date/version|Product discussion| +|------------|-----------------|-----------|-------------------------|-----------------------------------|------------------| +|Prometheus Metrics|`--experimental-enable-metrics`|The `/metrics` endpoint exposes metrics to be scraped by a Prometheus collector at regular intervals and stored for analysis.|We have yet to determine which metrics we want to expose and how|TBC|| +|Reduce Indexing Memory Usage|`--experimental-reduce-indexing-memory-usage`|Trades-off indexing speed with a lower RAM footprint.|We have yet to determine if a lot impacts performance and whether the RAM usage reduction is significant enough.|TBC|| + + +## 3.2. Runtime experimental features + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Feature nameHow to enableDescriptionMissing for stabilizationExpected stabilization date/versionProduct discussion
+ +``` +curl \ + -X PATCH 'http://localhost:7700/experimental-features/' \ + -H 'Content-Type: application/json' \ +--data-binary '{ + "vectorStore": true + }' +``` + + Enables storing and searching by using semantic vectorsConfidence in the speed of the indexation, search of the vectors, and API surfacev1.4 + + + +
+ +``` +curl \ + -X PATCH 'http://localhost:7700/experimental-features/' \ + -H 'Content-Type: application/json' \ +--data-binary '{ + "scoreDetails": true + }' +``` + + Enables computing detailed scoresConfidence on the computed details namesv1.4 + + + +
+ + + +## 4. Technical Details + +N/A + +## 5. Future Possibilities + +N/A \ No newline at end of file diff --git a/text/0194-experimental-feature-api.md b/text/0194-experimental-feature-api.md new file mode 100644 index 00000000..a7d75524 --- /dev/null +++ b/text/0194-experimental-feature-api.md @@ -0,0 +1,55 @@ +# Runtime experimental feature API + +## 1. Summary + +The runtime experimental feature API allows toggling the status of some [experimental features](./0193-experimental-features.md) at runtime. + +Due to its nature, this route itself is permanently experimental, in that the way of using it is not covered by [Meilisearch's stability guarantee](https://github.com/meilisearch/engine-team/blob/main/resources/versioning-policy.md). + +## 2. Motivation + +Historically, experimental features in the engine must be enabled from the CLI or via environment variables. + +The problem is that it requires restarting Meilisearch, so: + +- It induces downtime. +- Makes experimental features harder to enable for [Meilisearch Cloud](https://www.meilisearch.com/pricing?utm_campaign=oss&utm_source=engine&utm_medium=specifications) instances. + +The motivation of this feature is to remove these issues by allowing enabling and disabling experimental features at runtime. + +Due to the nature of some experimental features they might not be in scope for this API. The experimental features in scope for this API are called [*runtime* experimental features](./0193-experimental-features.md#32-runtime-experimental-features), while the ones not in scope are called [*instance* experimental features](./0193-experimental-features.md#31-instance-experimental-features). + +## 3. Functional Specification + +### 3.1 Routes + +Meilisearch exposes 2 routes to get or set the status of runtime experimental features. + +- GET `/experimental-features`: get the status of all the runtime experimental features. +- PATCH `/experimental-features`: set the status of some of the runtime experimental features. + +All routes return the status of the runtime experimental features after calling the route. + +This response is a JSON object containing the following fields: + +|Field name|Type|Experimental feature| +|----------|----|-----| +|`scoreDetails`|Boolean| [Score details](./0193-experimental-features.md#score-details) | +|`vectorStore`|Boolean| [Vector store](./0193-experimental-features.md#vector-store) | + +The PATCH routes accept as payload a JSON object containing the same fields as in the response, with the following effects on the corresponding feature: +- Setting a field to `true` enables the feature. +- Setting a field to `false` disables the feature. +- Setting a field to `null` or omitting a field leaves its value unchanged. + +### 3.2 Errors + +- 🔴 [`bad_request`](./0061-error-format-and-definitions.md#bad_request) for unknown fields in the payload or whenever a field is not a boolean. + +## 4. Technical Details + +N/A + +## 5. Future Possibilities + +N/A \ No newline at end of file diff --git a/text/0195-ranking-score.md b/text/0195-ranking-score.md new file mode 100644 index 00000000..b734c4e4 --- /dev/null +++ b/text/0195-ranking-score.md @@ -0,0 +1,183 @@ +# Ranking Score + +## 1. Summary + +Adds two kinds of scores to documents returned by a [search query](./0118-search-api.md). + +## 2. Motivation + +When configuring the Meilisearch relevancy according to their needs, users cannot know why one document has been favored over another. + +Showing how the documents ranked according to Meilisearch’s ranking rules unlocks: + +- Further customization of the developer workflow, such as fine-tuning settings and improving relevancy for example. +- Returning a unified list of results for multi-index search queries +- Sharding +- Debugging and helping users better understand how ranking works + +## 3. Functional Specification + +### 3.1. Ranking score + +A ranking score is a number attached to each document returned by a search when the [`showRankingScore`](./0118-search-api.md#3117-showrankingscore) flag is set to true in the search query. + +#### 3.1.1. Scale and interpretation + +The ranking score is contained between 1.0 and 0.0. A higher score signifies better relevancy, with 1.0 representing a perfect match, and 0.0 indicating that the document does not match the query (Meilisearch should not return documents that do not match the query). + +That number rates the relevancy of the document with respect to the specified search query and the current settings of the index. + +The score of a document follows its relevancy in the sense of Meilisearch, in that the first few ranking rules have a much higher influence on the score than the next rules. This is consistent with the way that later ranking rules are only used to break ties with earlier ranking rules, when ranking documents. + +#### 3.1.2. Score independence + +The score of a document is independent of what other documents are contained in the index but is influenced by the settings of the index. The table below details all the settings that can influence the score. Unlisted settings do not influence the ranking score. + +| Setting name | Influences if | Rationale | +|--------------|---------------|-----------| +|`searchableAttributes`|The `attribute` ranking rule is used|The `attribute` ranking rule rates the document depending on the attribute in which the query terms show up. The order is determined by `searchableAttributes`| +|`rankingRules`|Always|The score is computed by computing the subscore of each ranking rule with a weight that depends on their order.| +|`stopWords`|Always|Stop words influence the `words` ranking rule, which is almost always used| +|`synonyms`|Always|Synonyms influence the `words` ranking rule, which is almost always used| +|`typoTolerance`|The `typo` ranking rule is used|Used to compute the maximum number of typos for a query| + + +Additionally, the following can impact score independence: + +- If the `attribute` ranking rule is used, but `searchableAttributes` has not been specified, then the score is dependent on all the fields that appear in documents and their precise order, as determined by Meilisearch. +the score is dependent on the search query. + +Depending on the use case, it can be meaningful to compare scores coming from indexes with settings that are different: + +- When comparing two scores produced on two indexes with different settings, possibly on a distinct search query, one is comparing the relevancy of each of the scored documents to their respective search query. This is good to present the most relevant documents first when working with heterogeneous indexes, without taking into account which document best suits one single query. +- On the other hand, to find what document best suits one single query against two homogeneous indexes, one must be careful to make sure that the indexes have the settings above set to the same value. + +#### 3.1.3. The sort ranking rules do not impact the score + +Custom `sort` and `geosort` ranking rules modify the ranking of documents such that they are returned sorted by the value of the target field, rather than by their relevancy to the search query. + +As such, these ranking rules have no impact on the score. As a corollary of this, if a `sort` ranking rule is not the last ranking rule, then it is possible to see documents returned with ranking scores that are not monotonically decreasing. + +Similarly, re-ranking documents by their ranking score will ignore any `sort` ranking rule. + +If you need to factor sort ranking rules into your score, then use the [ranking score details](#32-ranking-score-details). + +### 3.2. Ranking score details + +(EXPERIMENTAL) The ranking score details are represented as an object attached to each document returned by a search when the [`showRankingScoreDetails`](./0118-search-api.md#3118-showrankingscoredetails) flag is set to true in the search query. + +The ranking score details are experimental and require enabling the corresponding [experimental feature](./0193-experimental-features.md#score-details). + +#### 3.2.1. General shape + +The fields of the object have for key the identifier of the various ranking rules that were applied, and for value an object with at least the following field: + +- `order`: the numerical order in which the ranking rule was applied. Starts at 0. Consecutive numbers denote ranking rules consecutively applied. + +Additionally, all ranking rules except the `sort` and `geosort` ranking rules have the following field: + +- `score`: the relevancy score of the document relative to this search query, for this ranking rule. A number between 1.0 and 0.0, with 1.0 meaning a perfect match to the query according to the ranking rule, and 0.0 no match. + +#### 3.2.2. Ranking-rule-specific fields + +Each ranking rule exposes specific fields meant to provide semantic information about how the ranking rule was applied to the document. + +The table below details these rule-specific fields. + +​ +| Ranking rule | Field description | +| :--------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `words` |
  • `matchingWords`: Number of words in the query that match in the document. The higher the better
  • `maxMatchingWords`: Maximum number of words in the query that can match in the document for this iteration of the `words` ranking rule. Usually, the query length, but if one of the query terms is set as a stop word, it won’t be counted here.
| +| `typo` |
  • `typoCount`: Number of typos to correct in the query so that the document matches for this iteration of the `typo` ranking rule.
  • `maxTypoCount`: Maximum number of typos possible in a document for this iteration of the `typo` ranking rule.
| +| `proximity` | No rule-specific field | +| `attribute` |
  • `attribute_ranking_order_score`: Results sorted based on the attribute ranking order
  • `query_word_distance_score`: Documents with attributes containing the query words close to their position in the query will be considered more relevant than documents containing the query words far from their position in the query
| +| `exactness` |
  • `matchType`: It has one of the following values:
    • `exactMatch`: The query exactly matches the entire value of an attribute
    • `matchesStart`: The query matches exactly the start of the value of an attribute
    • `noExactMatch`: The query doesn't exactly match a document
  • `matchingWords`: for `matchesStart`, the number of exact words contained in an attribute. The higher the better
  • `maxMatchingWords`: for `noExactMatch`, the maximum number of exact words contained in an attribute
| + +#### 3.2.3. Sort ranking rules + +`Sort` and `geosort` ranking rules appear as fields in the score details, but with the following difference: + +- Their key follows the following format: `{:attribute-sorted-on}:{:sort-direction}`, with the `:attribute-sorted-on` the name of the attribute that is being sorted on, and the `:sort-direction` either `asc` if the sort is in ascending order, or `desc` if the sort is in descending order. For the `geosort` ranking rule, it is similarly `_geoPoint({:lat}, {:lng}):{:sort-direction}`, with the `:lat` and `:lng` being the latitude and respective longitude of the point that serves as base to sort by distance. +- They don't have a `score` field, but instead they have a `value` field, representing the value used to sort the document. It is typically the value of the sorted attribute for the document, but can sometimes be a subvalue (case where the value is an array of values). +- For the `geosort`, there is an additional `distance` field representing the distance between the target point and the point used in the document to sort the document. + +#### 3.2.4 Example + +The following is an example of a `_scoreDetails` returned for a document matching a search query. + + +```json +"_rankingScoreDetails": { + "words": { + "order": 0, + "matchingWords": 1, + "maxMatchingWords": 1, + "score": 1 + }, + "typo": { + "order": 1, + "typoCount": 0, + "maxTypoCount": 1, + "score": 1 + }, + "proximity": { + "order": 2, + "score": 1 + }, + "attribute": { + "order": 3, + "attributes_ranking_order": 0.8333333333333334, + "attributes_query_word_order": 1, + "score": 0.8333333333333334 + }, + "exactness": { + "order": 4, + "matchType": "exactMatch", + "score": 1 + }, + "release_date:asc": { + "order": 5, + "value": 1165881600 + } +} +``` + +## 4. Technical Details + +### 4.1. Ranking score calculation + +The ranking score calculation in this section is given for informative purposes and is not normative. + +The implementation computes the [ranking score](#31-ranking-score) from each ranking rule (excluding `sort` and `geosort`) with two bits of data per ranking rule. For the `k`th applied ranking rule: + +- The maximum rank `max_rank_k` that a document can score with the rule, [independently from the other documents in the index](#312-score-independence) +- The rank `rank_k` of that document for that rule, with the highest rank being equal to the maximum rank, and the lowest rank being equal to 1. + +The score is given by the following formula, assuming `n` ranking rules denoted from `0` to `n-1`: + +``` +score = sum(i in 0..(n-1), (rank_i - 1) / product(j in 0..=i, max_rank_j)) + (rank_(n-1) / product(i in 0..n, max_rank_i)) +``` + +The intuition behind this formula is that every document falls in a range for each rule, between `rank_i / max_rank_i` and `(rank_i - 1) / max_rank_i`, and the next ranking rule allows to refine where the document is in this range, with the last ranking rule providing the exact score. + + +### 4.2. Hidden ranking rules + +If the [`displayedAttributes`](./0123-displayed-attributes-setting-api.md) list is defined, then attributes that are not part of that list, but are used in `sort` ranking rules are **hidden**. + +Instead of seeing `{:attribute-sorted-on}:{:sort-direction}` like described in [the relevant section](#323-sort-ranking-rules), the name of that field is replaced with ``, with `{:number}` a number that serves to uniquely distinguish between such hidden rules. + +Note: that number is not guaranteed to start at 0 nor to be consecutive. The only guarantee is that no hidden ranking rule will have the same number. + +Furthermore, the `value` that was used to sort the document is also hidden and replaced by `""`. + +### 4.3. Disabled optimization + +The engine optimizes search by skipping the application of ranking rules when there's only one remaining document (no tie to break). + +To compute an accurate score, however, all ranking rules must be applied, so this optimization is disabled as soon as a score is requested in the search request. When no scores are requested, the optimization is active. + +## 5. Future Possibilities + +- Extend the [multi-search API](./0192-multi-search-api.md) to rerank documents according to their score, providing federated search. diff --git a/text/0246-facet-search-api.md b/text/0246-facet-search-api.md new file mode 100644 index 00000000..e4418195 --- /dev/null +++ b/text/0246-facet-search-api.md @@ -0,0 +1,150 @@ +# Facet Search API + +## 1. Summary + +The facet search endpoint retrieves facet values from a query. + +## 2. Motivation + +When many values exist for a facet, end-users need to be able to discover non-show values they can select in order to refine their faceted search. Being able to search for values for a facet solves this need. + +## 3. Functional Specification + +Meilisearch exposes 1 route to perform facet search requests: + +- POST `indexes/:index_uid/facet-search` + +- 🔴 Sending an invalid index uid format for the `:index_uid` path parameter returns an [invalid_index_uid](0061-error-format-and-definitions.md#invalid_index_uid) error. +- 🔴 If the index does not exist, the API returns an [index_not_found](0061-error-format-and-definitions.md#index_not_found) error. + +If a master key is used to secure a Meilisearch instance, the auth layer returns the following errors: + +- 🔴 Accessing these routes without the `Authorization` header returns a [missing_authorization_header](0061-error-format-and-definitions.md#missing_authorization_header) error. +- 🔴 Accessing these routes with a key that does not have permissions (i.e. other than the master key) returns an [invalid_api_key](0061-error-format-and-definitions.md#invalid_api_key) error. + +`POST` HTTP verb errors: + +- 🔴 Omitting the Content-Type header returns a [missing_content_type](0061-error-format-and-definitions.md#missing_content_type) error. +- 🔴 Sending an empty Content-Type returns an [invalid_content_type](0061-error-format-and-definitions.md#invalid_content_type) error. +- 🔴 Sending a different Content-Type than `application/json` returns an [invalid_content_type](0061-error-format-and-definitions.md#invalid_content_type) error. +- 🔴 Sending an empty payload returns a [missing_payload](0061-error-format-and-definitions.md#missing_payload) error. +- 🔴 Sending an invalid JSON payload returns a [malformed_payload](0061-error-format-and-definitions.md#malformed_payload) error. + +### 3.1. Facet Search Payload Parameters + +| Field | Type | Required | +|-------------------------------------------------------|--------------------------|----------| +| [`facetName`](#311-facetName) | String | True | +| [`facetQuery`](#312-facetQuery) | String | False | + +#### 3.1.1. `facetName` + +- Type: String +- Required: True +- Default: `null` + +`facetName` contains the facet name to search values on. + +- 🔴 Omitting `facetName` returns a `missing_facet_search_facet_name`(0061-error-format-and-definitions.md#missing_facet_search_facet_name) error. +- 🔴 Sending a value with a different type than `String` for `facetName` returns a [missing_facet_search_facet_name](0061-error-format-and-definitions.md#minvalid_facet_search_facet_name) error. +- 🔴 Sending a field not defined as a `filterableAttributes` for `facetname` returns an [invalid_facet_search_facet_name](0061-error-format-and-definitions.md#invalid_facet_search_facet_name) error. + +#### 3.1.2. `facetQuery` + +- Type: String +- Required: False +- Default: `null` + +`facetQuery` contains the terms to search within the facet values of the `facetName`. + +> When `facetQuery` isn't specified, Meilisearch performs a **placeholder search**. A placeholder search returns all facet values for the searched facet, limited by the [maxValuesPerFacet index setting](157-faceting-setting-api.md#311-maxvaluesperfacet). +> It supports **prefix search** +> It supports **typo tolerance**. Modifying the [typo tolerance index setting](0117-typo-tolerance-setting-api.md) impacts the behavior of the facet search. + +> Meilisearch only considers one term for `facetQuery`. e.g if "harry potter" is typed the facet search consider "harry potter" as one single term. + +- 🔴 Sending a value with a different type than `String` or `null` for `facetQuery` returns an [invalid_facet_search_facet_query](0061-error-format-and-definitions.md#invalid_facet_search_facet_query) error. + +### 3.2. Additional Search Parameters + +Additional search parameters can be injected to the facet search query. If additional search parameters are set, the method will return facet values that both: + +- Match the facet query +- Are contained in the records matching the additional search parameters (`q`, `filter`, `matchingStrategy`) + +| Field | Type | Required | +|----------------------------------------------------------------|--------------------------|----------| +| [`q`](0118-search-api.md#311-q) | String | False | +| [`filter`](0118-search-api.md#312-filter) | String | False | +| [`matchingStrategy`](0118-search-api.md#3117-matchingstrategy) | String | False | + +- 🔴 These properties are validated as stipulated in their original specification and return the associated errors. + +> Non-listed search parameters are ignored if specified for the API call; no error is raised. + +### 3.3. Search Response Properties + +| Field | Type | Required | +|-------------------------------------------------|-----------------|----------| +| [`facetHits`](#331-facetHits) | Array[FacetHit] | True | +| [`facetQuery`](#332-facetQuery) | String | True | +| [`processingTimeMs`](#333-processingtimems) | Integer | True | + +#### 3.3.1. `facetHits` + +- Type: Array[FacetHit] +- Required: True + +Results of the facet search query as an array of facet hit. + +> FacetHit object represents a matched facet value for a facet search. + +##### 3.3.1.1. `facetHit` Object Properties + +`facetHit` sorting is controlled by the [sortFacetValuesBy index setting](157-faceting-setting-api.md#312-sortfacetvaluesby). + +| Field | Type | Required | +|----------------------------------------------|---------|----------| +| [`value`](#32112-formatted) | String | True | +| [`count`](#32111-geodistance) | Integer | True | + +###### 3.3.1.1.1. `value` + +- Type: String +- Required: True + +The facet value being matched. + +###### 3.3.1.1.2. `count` + +- Type: Integer +- Required: True + +The number of document containing the matched facet value. + +#### 3.3.2. `facetQuery` + +- Type: Integer +- Required: True + +Facet query originating the response. + +#### 3.3.3. `processingTimeMs` + +- Type: Integer +- Required: True + +Processing time of the facet search query in **milliseconds**. + +## 2. Technical Details + +### 2.1 Facet Containing Numbers Limitation + +- Numbers on the Meilisearch side are represented as [float64](https://en.wikipedia.org/wiki/Double-precision_floating-point_format) internally. When a user sends a number in a JSON document, it can be written in different ways but end up being the same number i.e. 12e4, 120000, 120000.0. There is an issue as to how to let the user find it if it is not written in the same way. +- Another issue is that float64 numbers sometimes lack precision. You can write a number in your document i.e. 12.6758493, search for this exact same number and don’t find it. The reason is that numbers are stored in a different way and an exact search will not find them. + +## 3. Future Possibilities + +- Add a property to customize the sort of returned facet values +- Highlight parts matching the facet value from the facet query +- Support for number search \ No newline at end of file diff --git a/text/157-faceting-setting-api.md b/text/157-faceting-setting-api.md index c4d97072..8ada5cb2 100644 --- a/text/157-faceting-setting-api.md +++ b/text/157-faceting-setting-api.md @@ -17,6 +17,7 @@ This settings will host the parameters to configure the faceting behavior for an | Field | Type | Required | |--------------------------------------------------|-----------------|----------| | [maxValuesPerFacet](#311-maxValuesPerFacet) | Integer | False | +| [sortFacetValuesBy](#312-sortFacetValuesBy) | Object | False | #### 3.1.1. `maxValuesPerFacet` @@ -34,6 +35,33 @@ Increasing this value can degrade performance as well as expose the data of an i The facets that are returned are sorted in ascending lexicographical order. +#### 3.1.2 `sortFacetValuesBy` + +- Type: Object +- Required: False +- Default: + +```json +{ + "*": "alpha" +} +``` + +Defines how facet values are sorted. By default, all facets (`*`) are sorted by name, alphanumerically in ascending order (`alpha`). + +It is possible to sort them by the number of documents containing a facet value in descending order using `count`. + +It is possible to specify a particular order for a facet. + +```json +{ + "*": "alpha", + "genre": "count" +} +``` + +In this example, values from facets other than `genres` will be displayed sorted by their name in ascending alphanumeric order, while values from the `genres` facet will be sorted in descending order by the count of the number of documents containing each value. + ## 3.2. API Endpoints Definition ### 3.2.1. Global Settings API Endpoints Definition @@ -54,7 +82,10 @@ Allow fetching the current definition of the faceting setting for an index. ```json { - "maxValuesPerFacet": 100 + "maxValuesPerFacet": 100, + "sortFacetValuesBy": { + "*": "alpha" + } } ``` @@ -103,8 +134,8 @@ See [Summarized `task` Object for `202 Accepted`](0060-tasks-api.md#summarized-t - 🔴 Sending an empty payload returns a [missing_payload](0061-error-format-and-definitions.md#missing_payload) error. - 🔴 Sending an invalid JSON payload returns a [malformed_payload](0061-error-format-and-definitions.md#malformed_payload) error. - 🔴 Sending an invalid index uid format for the `:index_uid` path parameter returns an [invalid_index_uid](0061-error-format-and-definitions.md#invalid_index_uid) error. -- 🔴 Sending a value different from `null` or with a different type than `Integer` for the `maxValuesPerFacet` field returns -an [invalid_settings_faceting](0061-error-format-and-definitions.md#invalid_settings_faceting) error. +- 🔴 Sending a value different from `null` or with a different type than `Integer` for the `maxValuesPerFacet` field returns an [invalid_settings_faceting](0061-error-format-and-definitions.md#invalid_settings_faceting) error. +- 🔴 Sending a value different from `null` or an object with value with a different type than `"alpha"` or `"count"` for the `sortFacetValuesBy` field returns an [invalid_settings_faceting](0061-error-format-and-definitions.md#invalid_settings_faceting) error. ###### 3.2.2.2.2.1. Async Errors @@ -149,5 +180,4 @@ The auth layer can return the following errors if Meilisearch is secured (a mast n/a ## 3. Future Possibilities - -- Introduces a field to define the sorting of facets. e.g `sort` by `count`/`alphanumeric` \ No newline at end of file +n/a