From 438d885d785e93621a9f62f40271be01de570008 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Wed, 4 Dec 2024 19:28:27 +0000 Subject: [PATCH] Update word-delimiter.md (#8883) Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> (cherry picked from commit a950b205898a9988aa8a2140e789a5e4c1a7e41e) Signed-off-by: github-actions[bot] --- _analyzers/token-filters/word-delimiter.md | 1 + 1 file changed, 1 insertion(+) diff --git a/_analyzers/token-filters/word-delimiter.md b/_analyzers/token-filters/word-delimiter.md index 3ef5d12024..d820fae2a0 100644 --- a/_analyzers/token-filters/word-delimiter.md +++ b/_analyzers/token-filters/word-delimiter.md @@ -10,6 +10,7 @@ nav_order: 470 The `word_delimiter` token filter is used to split tokens at predefined characters and also offers optional token normalization based on customizable rules. We recommend using the `word_delimiter_graph` filter instead of the `word_delimiter` filter whenever possible because the `word_delimiter` filter sometimes produces invalid token graphs. For more information about the differences between the two filters, see [Differences between the `word_delimiter_graph` and `word_delimiter` filters]({{site.url}}{{site.baseurl}}/analyzers/token-filters/word-delimiter-graph/#differences-between-the-word_delimiter_graph-and-word_delimiter-filters). +{: .important} The `word_delimiter` filter is used to remove punctuation from complex identifiers like part numbers or product IDs. In such cases, it is best used with the `keyword` tokenizer. For hyphenated words, use the `synonym_graph` token filter instead of the `word_delimiter` filter because users frequently search for these terms both with and without hyphens. {: .note}