uk4b Transformers

This release introduces a REPL for models trained for my and @dchaplinsky's paper on GPT-2 Metadata Pretraining Towards Instruction Finetuning for Ukrainian.

The REPL is accessible via a new CLI program, hat.

To use hat, first install some additional dependencies and models:

pip install haloop --upgrade               # make sure you have at least 0.0.7
pip install bitsandbytes sentencepiece     # I opted for not installing these as dependencies for now

wget https://a.wilab.org.ua/gpt/wiki.model  # sentencepiece tokenizer
wget https://a.wilab.org.ua/gpt/ckpt10m.pt  # model checkpoint for GPT-2 Large

Now, start the REPL:

hat --spm wiki.model ckpt10m.pt