[AutoParallel] BF16-o1/FP16-o1 PASS support training and generation

[zhaoyinglia]

PR types

Others

PR changes

Others

Describe

• Support AutoParallel BF16-o1 and FP16-o1 in the training and generation scene.
• Support Engine dtype convert when the dtype is different between ckpt and current model
• Add dist_ops to support input_dist_attr is same with output_dist_attr.
• Fix the conflict between AMP PASS and Sharding Pass
• Interface of amp

        strategy = auto.Strategy()
        amp = strategy.amp
        amp.dtype = "float16"  # or "bfloat16"
        amp.level = "o1"
        amp.custom_white_list = ['softmax', 'layer_norm', 'gelu']
        amp.custom_black_list = [
            'c_softmax_with_cross_entropy',
            'elementwise_div',
            'reduce_sum',
        ]

[paddle-bot]

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[JZ-LIANG]

LGTM