Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Tools] Add convert tools in xft. #78

Merged
merged 1 commit into from
Nov 24, 2023
Merged

[Tools] Add convert tools in xft. #78

merged 1 commit into from
Nov 24, 2023

Conversation

Duyi-Wang
Copy link
Contributor

Usage:

import xfastertransformer as xft
converter = xft.LlamaConvert()
converter("/data/llama-2-7b-chat","/data/llama-2-7b-chat-xft", "fp16")

Params:input_dir, output_dir=None, dtype: str = "fp16", processes=8
if output_dir is None, default use input_dir with a suffix -xft, like:

converter("/data/llama-2-7b-chat")
# use output_dir = "/data/llama-2-7b-chat-xft"

Support converter list:

LlamaConvert
ChatGLMConvert
ChatGLM2Convert
OPTConvert
BaichuanConvert

TODO:

  • README
  • Lazy import
  • hugging face dependency version

@changqi1
Copy link
Contributor

python -c "import xfastertransformer as xft; xft.LlamaConvert().converter("/data/llama-2-7b-chat","/data/llama-2-7b-chat-xft", "fp16")"

@changqi1 changqi1 merged commit 91f09e0 into intel:main Nov 24, 2023
1 check passed
@Duyi-Wang Duyi-Wang deleted the tools branch November 24, 2023 08:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants