Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GPT-JT model #11

Open
Ayushk4 opened this issue Mar 20, 2023 · 0 comments
Open

Add GPT-JT model #11

Ayushk4 opened this issue Mar 20, 2023 · 0 comments
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed

Comments

@Ayushk4
Copy link
Member

Ayushk4 commented Mar 20, 2023

Following steps would be required:

  • Place model file, vocab.json, added_tokens.json, config.json from togethercomputer/GPT-JT-6B-v1 in a directory <modelDirectory>.

  • Convert model from torch to a serialized format: Run cformers/cformers/cpp/converters/convert_gptj_to_ggml.py with python3 convert_gptj_to_ggml.py <modelDirectory> 1 --- for bloom/GPTNeoX before <modelDirectory> you must add Huggingface Model Card (for example EleutherAI/pythia-6.9b-deduped)

  • Convert serialized model to quantized format: cformers/cformers/cpp/quantize_gptj.cpp as cd cformers/cformers/cpp && make && ./quantize_gptj <modelDirectory>/ggml-gptj-6b-model.bin <modelDirectory>/int4_fixed_zero.bin 2

  • Update Readme and cformers/cformers/interface.py with links and mention GPT-JT is supported

  • Test it, share how well it works and shoot a PR.

@Ayushk4 Ayushk4 added good first issue Good for newcomers enhancement New feature or request help wanted Extra attention is needed labels Mar 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant