This repository has been archived by the owner on Jul 4, 2023. It is now read-only.
Add BPE Text Encoder #7
Labels
enhancement
New feature or request
good first issue
Good for newcomers
help wanted
Extra attention is needed
Tokenizer Reference: https://github.com/eladhoffer/seq2seq.pytorch/blob/master/seq2seq/tools/tokenizer.py
The text was updated successfully, but these errors were encountered: