We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Currently, mlc-serve takes prompts in two paths: request.mesasges and request.debug_options.prompt. https://github.com/octoml/mlc-llm/blob/batch-serving/serve/mlc_serve/engine/engine_common.py#L43
mlc-serve
request.mesasges
request.debug_options.prompt
The expected behavior is not clear until you deep-dive the codebase. Better way can be have a flag to disable the application of chat template.
The text was updated successfully, but these errors were encountered:
Add model weight variant in iOS (octoml#187)
da63f2c
Co-authored-by: Yaxing Cai <[email protected]>
No branches or pull requests
Currently,
mlc-serve
takes prompts in two paths:request.mesasges
andrequest.debug_options.prompt
.https://github.com/octoml/mlc-llm/blob/batch-serving/serve/mlc_serve/engine/engine_common.py#L43
The expected behavior is not clear until you deep-dive the codebase.
Better way can be have a flag to disable the application of chat template.
The text was updated successfully, but these errors were encountered: