Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement the composition of cos_double_grad #62340

Merged
merged 6 commits into from
Mar 14, 2024
Merged

Implement the composition of cos_double_grad #62340

merged 6 commits into from
Mar 14, 2024

Conversation

YibinLiu666
Copy link
Contributor

@YibinLiu666 YibinLiu666 commented Mar 3, 2024 •
edited
Loading

PR types

Others

PR changes

Others

Description

实现pow的一阶微分组合，测试代码为：

import paddle
import numpy as np

def test_cos_double_grad():
    paddle.framework.core.set_prim_eager_enabled(False)
    shape = [1024, 512]
    x = paddle.to_tensor(np.random.randn(*shape), dtype="float32")
    x.stop_gradient=False
    y_fused_res = paddle.cos(x)
    dx = paddle.grad(y_fused_res, x, create_graph=True, retain_graph=False)[0]
    ddx_fused_res = paddle.grad(dx, x, create_graph=False, retain_graph=False)[0]
    paddle.framework.core.set_prim_eager_enabled(True)
    paddle.framework.core._set_prim_backward_blacklist("cos_grad")
    y_composite_res = paddle.cos(x)
    dx = paddle.grad(y_composite_res, x, create_graph=True, retain_graph=False)[0]
    ddx_composite_res = paddle.grad(dx, x, create_graph=False, retain_graph=False)[0]
    np.testing.assert_allclose(ddx_fused_res.numpy(), ddx_composite_res.numpy(), 1e-6, 1e-6)

@HydrogenSulfate

@paddle-bot Paddle-bot
Copy link

paddle-bot bot commented Mar 3, 2024

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@paddle-bot paddle-bot bot added the contributor External developers label Mar 3, 2024
HydrogenSulfate
HydrogenSulfate reviewed Mar 4, 2024
View reviewed changes
paddle/fluid/prim/api/composite_backward/composite_double_backward_api.h Outdated
@@ -53,6 +53,29 @@ void tanh_double_grad(const Tensor& out,
}
}

template <typename T>
void cos_double_grad(const Tensor& x,
const paddle::optional<Tensor>& grad_out,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

grad_out即dy会是optional吗？dy应该是一阶算子的一个输入，如果用户不在grad接口里指定dy，那dy默认会是全1矩阵而不会是空的吧

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Paddle/paddle/phi/kernels/impl/activation_grad_impl.h

Line 670 in bdd1fe8

void SinDoubleGradKernel(const Context& dev_ctx,
，这里大算子的实现是optional，因此如果组合实现是optional的编译的时候会报错。

@YibinLiu666 YibinLiu666 mentioned this pull request Mar 6, 2024
Closed
28 tasks
Charles-hit
Charles-hit reviewed Mar 12, 2024
View reviewed changes
test/prim/prim/vjp/eager/test_comp_eager_cos_double_grad.py Outdated
x = paddle.to_tensor(primal, dtype='float32', stop_gradient=False)
x.stop_gradient = False
y = paddle.cos(x)
x_cotangent = paddle.grad(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这儿测试的是一阶，需要加二阶测试

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Charles-hit
Charles-hit reviewed Mar 12, 2024
View reviewed changes
test/prim/prim/vjp/eager/test_comp_eager_cos_double_grad.py Outdated
x = paddle.to_tensor(primal, dtype='float32', stop_gradient=False)
x.stop_gradient = False
y = paddle.cos(x)
x_cotangent = paddle.grad(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

同上

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

HydrogenSulfate
HydrogenSulfate reviewed Mar 12, 2024
View reviewed changes
test/prim/prim/vjp/eager/test_comp_eager_cos_double_grad.py Outdated
import paddle
from paddle.base import core

sys.path.append('../../../../legacy_test/')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个应该可以删除掉

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Charles-hit
Charles-hit previously approved these changes Mar 13, 2024
View reviewed changes
heavyrain-lzy
heavyrain-lzy previously approved these changes Mar 13, 2024
View reviewed changes
Copy link
Contributor

@heavyrain-lzy heavyrain-lzy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Charles-hit Charles-hit merged commit 706ba85 into PaddlePaddle:develop Mar 14, 2024
30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@HydrogenSulfate HydrogenSulfate HydrogenSulfate left review comments

@Charles-hit Charles-hit Charles-hit approved these changes

@heavyrain-lzy heavyrain-lzy heavyrain-lzy approved these changes

Assignees
No one assigned
Labels
contributor External developers
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@YibinLiu666 @HydrogenSulfate @heavyrain-lzy @Charles-hit