Add GPT-JT model #11
Labels
enhancement
New feature or request
good first issue
Good for newcomers
help wanted
Extra attention is needed
Following steps would be required:
Place model file, vocab.json, added_tokens.json, config.json from togethercomputer/GPT-JT-6B-v1 in a directory
<modelDirectory>
.Convert model from torch to a serialized format: Run cformers/cformers/cpp/converters/convert_gptj_to_ggml.py with
python3 convert_gptj_to_ggml.py <modelDirectory> 1
--- for bloom/GPTNeoX before<modelDirectory>
you must add Huggingface Model Card (for exampleEleutherAI/pythia-6.9b-deduped
)Convert serialized model to quantized format: cformers/cformers/cpp/quantize_gptj.cpp as
cd cformers/cformers/cpp && make && ./quantize_gptj <modelDirectory>/ggml-gptj-6b-model.bin <modelDirectory>/int4_fixed_zero.bin 2
Update Readme and cformers/cformers/interface.py with links and mention GPT-JT is supported
Test it, share how well it works and shoot a PR.
The text was updated successfully, but these errors were encountered: