Skip to content

Commit

Permalink
Update word-delimiter.md (#8883)
Browse files Browse the repository at this point in the history
Signed-off-by: kolchfa-aws <[email protected]>
(cherry picked from commit a950b20)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
  • Loading branch information
github-actions[bot] committed Dec 4, 2024
1 parent c195990 commit 438d885
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions _analyzers/token-filters/word-delimiter.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ nav_order: 470
The `word_delimiter` token filter is used to split tokens at predefined characters and also offers optional token normalization based on customizable rules.

We recommend using the `word_delimiter_graph` filter instead of the `word_delimiter` filter whenever possible because the `word_delimiter` filter sometimes produces invalid token graphs. For more information about the differences between the two filters, see [Differences between the `word_delimiter_graph` and `word_delimiter` filters]({{site.url}}{{site.baseurl}}/analyzers/token-filters/word-delimiter-graph/#differences-between-the-word_delimiter_graph-and-word_delimiter-filters).
{: .important}

The `word_delimiter` filter is used to remove punctuation from complex identifiers like part numbers or product IDs. In such cases, it is best used with the `keyword` tokenizer. For hyphenated words, use the `synonym_graph` token filter instead of the `word_delimiter` filter because users frequently search for these terms both with and without hyphens.
{: .note}
Expand Down

0 comments on commit 438d885

Please sign in to comment.