New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

输入输出格式问题 #48

Open

zixiliuUSC opened this issue Jul 25, 2022 · 1 comment

zixiliuUSC commented Jul 25, 2022

因为发现在上一个同样问题里面问了没有回答，所以再开一个新的issue。
求问作者，你预训练的时候应该没有调用T5原生的_shift_right函数，是吗？因为如果调用了的话，解码端实际的开头是[PAD]；没调用的话，解码端预训练的模式是：输入[CLS]XXXXXXXX，输出XXXXXXX[SEP]。这样理解对吧？

Owner

renmada commented Jul 27, 2022

输入和输入都是[CLS]XXXXXXXX[SEP]

https://github.com/ZhuiyiTechnology/t5-pegasus/blob/a5211b7d4c6a4d92973eca57d9dcfc0cb0774662/finetune.py#L65

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment