Disable fused causal attention

[tianleiwu]

Description

There is accuracy regression in GPT-2 model. Top1 match rate (vs PyTorch model) drops about 1%. The cause is the fused causal attention uses fp16 accumulation. Disable it by default and user could use an environment variable ORT_ENABLE_FUSED_CAUSAL_ATTENTION=1 to turn on it manually.

It also updated the GPT-2 parity test script to generate left side padding to reflect the actual usage.

To test:

python -m onnxruntime.transformers.models.gpt2.convert_to_onnx -m gpt2 --output gpt2.onnx -o -p fp16 --use_gpu

The top1-match-rate in the output is on-par with ORT 1.13.1.

Motivation and Context

[yufenglee]

I'm not sure it is a good idea to disable fMHA by defaut.
The accuracy check is based on dummy inputs which are meaningless. Intuitively, it tends to generate logits that are more neural to all tokens, i.e., no/less preference on next token. I would say it'd better to randomly select 1000 real(meaning) sentences as the test data set.
In addition, fMHA is only enabled for the context input, not for all iterations.

[tianleiwu]

I'm not sure it is a good idea to disable fMHA by defaut. The accuracy check is based on dummy inputs which are meaningless. Intuitively, it tends to generate logits that are more neural to all tokens, i.e., no/less preference on next token. I would say it'd better to randomly select 1000 real(meaning) sentences as the test data set. In addition, fMHA is only enabled for the context input, not for all iterations.

Good suggestion. We will improve the test with real sentences, and re-evaluate this later.
Based on current test result, I think it is better to turn off it by default. Even though 1% drop is small, it is still regression on 1.13.