Add limit token filter docs #8159
Conversation
Signed-off-by: Anton Rubin <[email protected]>
Signed-off-by: Anton Rubin <[email protected]>
AntonEliatra
requested review from
kolchfa-aws,
Naarcha-AWS,
vagimeli,
AMoo-Miki,
natebower,
dlvenable,
stephen-crawford and
epugh
as code owners
|
Thank you for submitting your PR. The PR states are In progress (or Draft) -> Tech review -> Doc review -> Editorial review -> Merged. Before you submit your PR for doc review, make sure the content is technically accurate. If you need help finding a tech reviewer, tag a maintainer. When you're ready for doc review, tag the assignee of this PR. The doc reviewer may push edits to the PR directly or leave comments and editorial suggestions for you to address (let us know in a comment if you have a preference). The doc reviewer will arrange for an editorial review. |
|
@udabhas Will you review this PR for technical accuracy? Thank you. |
vagimeli
added
3 - Tech review
Signed-off-by: Anton Rubin <[email protected]>
Signed-off-by: AntonEliatra <[email protected]>
@udabhas @varun-lodaya This is a month old. We need tech review approval to move it forward in the documentation process. Please review this week or provide a peer who can review it. Thank you. |
Signed-off-by: Anton Rubin <[email protected]>
Signed-off-by: Fanit Kolchina <[email protected]>
kolchfa-aws
added
5 - Editorial review
| :--- | :--- | :--- | :--- | ||
| `max_token_count` | Optional | Integer | The maximum number of tokens to be generated. Default is `1`. | ||
| `consume_all_tokens` | Optional | Boolean | (Expect level setting) Use all tokens from tokenizer, even if result exceeds `max_token_count`. When this parameter is set, the output still only contains the number of tokens specified in `max_token_count`. However, all tokens generated by the tokenizer are processed. Default is `false`. | ||
|
|
There was a problem hiding this comment.
Line 19: What does "(Expect level setting)" mean?
There was a problem hiding this comment.
Expert 😄 Thanks for the catch.
There was a problem hiding this comment.
YW, but should be "Expert-level setting" in that case 😄
Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: kolchfa-aws <[email protected]>
Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: kolchfa-aws <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>
* adding limit token filter docs Signed-off-by: Anton Rubin <[email protected]> * removing parameter which is not working #8153 Signed-off-by: Anton Rubin <[email protected]> * adding consume_all_tokens Signed-off-by: Anton Rubin <[email protected]> * Update limit.md Signed-off-by: AntonEliatra <[email protected]> * updating parameter table Signed-off-by: Anton Rubin <[email protected]> * Doc review Signed-off-by: Fanit Kolchina <[email protected]> * Apply suggestions from code review Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: kolchfa-aws <[email protected]> * Update _analyzers/token-filters/index.md Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: kolchfa-aws <[email protected]> --------- Signed-off-by: Anton Rubin <[email protected]> Signed-off-by: AntonEliatra <[email protected]> Signed-off-by: Fanit Kolchina <[email protected]> Signed-off-by: kolchfa-aws <[email protected]> Co-authored-by: Fanit Kolchina <[email protected]> Co-authored-by: kolchfa-aws <[email protected]> Co-authored-by: Nathan Bower <[email protected]> (cherry picked from commit 1026a84) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Description
Add limit token filter docs
Issues Resolved
Closes #8153
Version
all
Checklist
For more information on following Developer Certificate of Origin and signing off your commits, please check here.