Pegasus Documentation May Conflict With Seq2Seq ReadMe

[kingpalethe]

Here, under tips and tricks.....
https://github.com/huggingface/transformers/blob/master/examples/seq2seq/README.md#tips-and-tricks
Both finetuning and eval are 30% faster with --fp16. For that you need to install apex.

But in the documentation...
https://huggingface.co/transformers/master/model_doc/pegasus.html#examples
FP16 is not supported (help/ideas on this appreciated!).

Also in the documentation
https://huggingface.co/transformers/master/model_doc/pegasus.html#examples
Script to fine-tune pegasus on the XSUM dataset.
leads to a 404: https://github.com/huggingface/transformers/blob/master/examples/seq2seq/finetune_pegasus_xsum.sh

[patil-suraj]

Hi @kingpalethe,

In general, for BART and Marian models, training and eval is faster with fp16, except Pegasus and T5 which currently don't work well with fp16

Yes, the fine-tuning script is now moved under examples/research_projects/seq2seq-distillation dir,
https://github.com/huggingface/transformers/tree/master/examples/research_projects/seq2seq-distillation

Thanks for reporting,

Also please note that this script is not maintained anymore and is provided as-is. We only maintain the finetune_trainer.py script now.

[github-actions]

This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.

[555296]

#26521 @[email protected]#v0

[555296]

sio