New functions added #13
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* qwen layernorm as input * add group size
* add env to disable compile opt * fix style * fix style
* add chatglm fuse mlp
* Add initial support for LNL nightly performance tests * Small fix
* update benchmark readme update new comment with memory usage included * Update README.md
* Limit trl version in example * Limit trl version in example
* support minicpm 1b & qwen 1.5b gw * support minicpm 1b * support minicpm 2b * fix style & error * fix style & update * remove print
* Add dummy model in iGPU perf * Add dummy model in iGPU perf * Fix
* replace gradio_web_server.patch to adjust webui * fix patch problem --------- Co-authored-by: ATMxsp01 <[email protected]>
* llama 3.1/3.2 support compresskv * update * fix transformers 4.45 error * fix style * fix typo * disable llama3.2 1b compresskv
* fix three issues * limit mixed_precision for CW only
* Add initial support for llama3.2-1b/3b * move llama3.2 support into current llama_mp impl
* add minicpm npu * optimize model
* change inter_pp * add comment
* update Readme for FastChat docker demo * update readme * add 'Serving with FastChat' part in docs * polish docs --------- Co-authored-by: ATMxsp01 <[email protected]>
* add ollama troubleshoot en * zh ollama troubleshoot * llamacpp trouble shoot * llamacpp trouble shoot * fix * save gpu memory
…12564) * Add --modelscope for more models * minicpm --------- Co-authored-by: ATMxsp01 <[email protected]>
* Update Dockerfile * Update Dockerfile * Update start-vllm-service.sh
…l2 (#12583) * Add --modelscope option for glm-v4 and MiniCPM-V-2_6 * glm-edge * minicpm-v-2_6:don't use model_hub=modelscope when use lowbit; internvl2 --------- Co-authored-by: ATMxsp01 <[email protected]>
* add npu support for baichuan * Update baichuan_mp.py * Update baichuan_mp.py
* add compresskv back for mistral * fix * fix
* Update open webui doc * Resolve comments
* fix npu save * update
* Update baichuan2.py * style fix
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
1. Why the change?
2. User API changes
3. Summary of the change
4. How to test?
1234). And paste your action link here once it has been successfully finished.5. New dependencies
- Dependency1
- Dependency2
- ...
- Dependency1 and license1
- Dependency2 and license2
- ...