New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

似乎这是一个 uncased 模型，对英文大写和空格等 token 进行了处理，对于中文来说 cased version 是否更合适？ #60

Open

qiguanqiang opened this issue Mar 28, 2023 · 1 comment

qiguanqiang commented Mar 28, 2023

我是一个正在学习生成式模型的小白，请大佬不吝赐教

Owner

renmada commented Apr 3, 2023

中文的类bert tokenizer 都是uncased。
个人觉得英文是关注cased的情况比较多

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment