update submodules (#118)

* update submodules * update * update llama.cpp docs * update links in readme_CN to modelscope
rubra-ai · Jul 3, 2024 · c9433a7 · c9433a7
1 parent de619d5
commit c9433a7
Show file tree

Hide file tree

Showing 4 changed files with 17 additions and 9 deletions.
diff --git a/README_CN.md b/README_CN.md
@@ -16,13 +16,13 @@ Rubra 增强了当前最流行的一系列开放权重大模型（LLM）的工
 
 | 模型 | 最大上下文长度 | 大小 | 基础模型发布者 |
 |---------------------------------------------------------------|----------------|------|----------------------|
-| [rubra-ai/Meta-Llama-3-8B-Instruct](https://huggingface.co/rubra-ai/Meta-Llama-3-8B-Instruct)   | 8,000          | 8B   | Meta             |
-| [rubra-ai/Meta-Llama-3-70B-Instruct](https://huggingface.co/rubra-ai/Meta-Llama-3-70B-Instruct) | 8,000          | 70B  | Meta             |
-| [rubra-ai/gemma-1.1-2b-it](https://huggingface.co/rubra-ai/gemma-1.1-2b-it)                     | 8,192          | 2B   | Google                 |
-| [rubra-ai/Mistral-7B-Instruct-v0.3](https://huggingface.co/rubra-ai/Mistral-7B-Instruct-v0.3)   | 32,000         | 7B   | Mistral              |
-| [rubra-ai/Mistral-7B-Instruct-v0.2](https://huggingface.co/rubra-ai/Mistral-7B-Instruct-v0.2)   | 32,000         | 7B   | Mistral              |
-| [rubra-ai/Phi-3-vision-128k-instruct](https://huggingface.co/rubra-ai/Phi-3-vision-128k-instruct)| 128,000        | 3B   | Microsoft              |
-| [rubra-ai/Qwen2-7B-Instruct](https://huggingface.co/rubra-ai/Qwen2-7B-Instruct)                 | 131,072        | 7B   | Qwen                   |
+| [rubra-ai/Meta-Llama-3-8B-Instruct](https://www.modelscope.cn/models/rubraAI/Meta-Llama-3-8B-Instruct)   | 8,000          | 8B   | Meta             |
+| [rubra-ai/Meta-Llama-3-70B-Instruct](https://www.modelscope.cn/models/rubraAI/Meta-Llama-3-70B-Instruct) | 8,000          | 70B  | Meta             |
+| [rubra-ai/gemma-1.1-2b-it](https://www.modelscope.cn/models/rubraAI/Gemma-1.1-2b-Instruct)                     | 8,192          | 2B   | Google                 |
+| [rubra-ai/Mistral-7B-Instruct-v0.3](https://www.modelscope.cn/models/rubraAI/Mistral-7B-Instruct-v0.3)   | 32,000         | 7B   | Mistral              |
+| [rubra-ai/Mistral-7B-Instruct-v0.2](https://www.modelscope.cn/models/rubraAI/Mistral-7B-Instruct-v0.2)   | 32,000         | 7B   | Mistral              |
+| [rubra-ai/Phi-3-vision-128k-instruct](https://www.modelscope.cn/models/rubraAI/Phi-3-mini-128k-instruct)| 128,000        | 3B   | Microsoft              |
+| [rubra-ai/Qwen2-7B-Instruct](https://www.modelscope.cn/models/rubraAI/Qwen2-7B-Instruct)                 | 131,072        | 7B   | Qwen                   |
 
 ## Demo
 

diff --git a/docs/docs/inference/llamacpp.mdx b/docs/docs/inference/llamacpp.mdx
@@ -52,6 +52,14 @@ For example:
 wget https://huggingface.co/rubra-ai/Meta-Llama-3-8B-Instruct-GGUF/resolve/main/rubra-meta-llama-3-8b-instruct.Q8_0.gguf
 ```
 
+:::info
+For large multi-part model files, such as [rubra-meta-llama-3-70b-instruct_Q6_K-0000*-of-00003.gguf](https://huggingface.co/rubra-ai/Meta-Llama-3-70B-Instruct-GGUF/tree/main), use the following command to merge them before proceeding to the next step:
+```
+./llama-gguf-split --merge rubra-meta-llama-3-70b-instruct_Q6_K-0000*-of-00003.gguf rubra-meta-llama-3-70b-instruct_Q6_K.gguf
+```
+This will merge multi-part model files to one gguf file `rubra-meta-llama-3-70b-instruct_Q6_K.gguf`.
+:::
+
 ### 5. Start the OpenAI Compatible Server
 
 ```bash

diff --git a/rubra-tools b/rubra-tools
diff --git a/tools.cpp b/tools.cpp