adds new CPU kernel for SGD op supporting BF16 data type

[arogowie-intel]

PR types

New features

PR changes

OPs

Describe

This PR adds new CPU kernel for SGD op supporting BF16 data type.

[paddle-bot-old]

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[arogowie-intel]

@wozna @arlesniak @jczaja Could you start review please?

[wozna]

Everything I mentioned is about PADDLE_ENFORCE
On this site, there is information on how we should write error messages: https://github.com/PaddlePaddle/Paddle/wiki/Paddle-Error-Message-Writing-Specification-(English-Verison)

[arlesniak]

Please make integration test i.e. test_fit_a_line.py to verify this implementation (it can be done in next PR)

[arogowie-intel]

@wozna, @arlesniak Please review again.

[arogowie-intel]

@luotao1
The PR-CI-APPROVAL fails because of:
1.

0. Unittest is not allowed to be disabled.

I'm using

@unittest.skipIf(not core.supports_bfloat16(),

To not run tests on platforms which does not support bfloat16 data type.

1. Developers are not allowed to set the check_dygraph field directly, which is set to True by default.

I'm turning off dygraph mode since we does not support it with bfloat16 yet.

[arogowie-intel]

@luotao1 Could you help with PR-CI-ROCM-Compile?
It somehow fails during compilation, however locally I do not see such an error. Moreover no other CI run failed with such an error.

[luotao1]

@qili93 Could you help see error of PR-CI-ROCM-Compile?

[luotao1]

@arogowie-intel You can see the ROCM build step in https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/guides/rocm_docs/paddle_install_cn.html

[qili93]

Hi @arogowie-intel to fix the build error or "PR-CI-ROCM-Compile", please change your code as following:

 PADDLE_ENFORCE(grad_var->IsType<framework::SelectedRows>(),
# ------ change the above code to below ----------
 PADDLE_ENFORCE_EQ(grad_var->IsType<framework::SelectedRows>(), true, 

ROCM defined PADDLE_ENFORCE as a device function in enforce.h, so please avoid to use it in host functions.

2021-04-12 21:28:13 In file included from /paddle/paddle/fluid/operators/optimizers/sgd_op.cu:16:
2021-04-12 21:28:13 /paddle/paddle/fluid/operators/optimizers/sgd_op.h:250:22: error: expected ')'
2021-04-12 21:28:13                      platform::errors::InvalidArgument(
2021-04-12 21:28:13                      ^
2021-04-12 21:28:13 /paddle/paddle/fluid/operators/optimizers/sgd_op.h:249:7: note: to match this '('
2021-04-12 21:28:13       PADDLE_ENFORCE(grad_var->IsType<framework::SelectedRows>(),
2021-04-12 21:28:13       ^
2021-04-12 21:28:13 /paddle/paddle/fluid/platform/enforce.h:438:13: note: expanded from macro 'PADDLE_ENFORCE'
2021-04-12 21:28:13       printf("Error: %s:%d Assertion `%s` failed. " __FORMAT "\n", __FILE__, \
2021-04-12 21:28:13             ^

[arlesniak]

LGTM

[arogowie-intel]

@luotao1 Could you please start your review?

[arogowie-intel]

@luotao1 I'm waiting for the last CI pm-test run, which is in "Pending" status from yesterday. Moreover it is not required one. Regarding the PR-CI-APPROVAL I have explained why it is failing in my earlier comment.
Could you start your review please? This PR is quite important for continuation of word2vec training work.

[luotao1]

I'm waiting for the last CI pm-test run, which is in "Pending" status from yesterday. Moreover it is not required one.

@arogowie-intel You can ignore the CI which is not the required one.