Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement the composition of cos_double_grad #62340

Merged
merged 6 commits into from
Mar 14, 2024
Merged

Implement the composition of cos_double_grad #62340

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
a8f36be
Implement the composition of cos_double_grad
YibinLiu666 Mar 3, 2024
57728c1
not use optional
YibinLiu666 Mar 4, 2024
9327a86
add test
YibinLiu666 Mar 9, 2024
8f3e01f
update
YibinLiu666 Mar 9, 2024
adb7d51
update test
YibinLiu666 Mar 12, 2024
c917b6f
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
YibinLiu666 Mar 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions paddle/fluid/prim/api/composite_backward/composite_double_backward_api.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,29 @@ void tanh_double_grad(const Tensor& out,
}
}

template <typename T>
void cos_double_grad(const Tensor& x,
const paddle::optional<Tensor>& grad_out,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

grad_out即dy会是optional吗？dy应该是一阶算子的一个输入，如果用户不在grad接口里指定dy，那dy默认会是全1矩阵而不会是空的吧

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Paddle/paddle/phi/kernels/impl/activation_grad_impl.h

Line 670 in bdd1fe8

void SinDoubleGradKernel(const Context& dev_ctx,
，这里大算子的实现是optional，因此如果组合实现是optional的编译的时候会报错。

const Tensor& grad_x_grad,
Tensor* x_grad,
Tensor* grad_out_grad) {
// cos grad grad : ddout = -sinx * ddx, dx = -dy * cosx * ddx
if (x_grad) {
if (grad_out) {
auto x_grad_tmp = -(grad_out.get() * cos<T>(x) * grad_x_grad);
set_output<T>(x_grad_tmp, x_grad);
} else {
auto x_grad_tmp = full<T>(common::vectorize(x.dims()), 0.0, x.dtype());
set_output<T>(x_grad_tmp, x_grad);
}
}

if (grad_out_grad) {
auto grad_out_grad_tmp = -sin<T>(x) * grad_x_grad;
set_output<T>(grad_out_grad_tmp, grad_out_grad);
}
}

template <typename T>
void tanh_triple_grad(const Tensor& out,
const Tensor& grad_out_forward,
Expand Down
1 change: 1 addition & 0 deletions paddle/phi/api/yaml/backward.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -436,6 +436,7 @@
optional: grad_out
backward : cos_triple_grad
inplace : (grad_x_grad -> grad_out_grad)
composite : cos_double_grad(x, grad_out, grad_x_grad, x_grad, grad_out_grad)

- backward_op : cos_grad
forward : cos (Tensor x) -> Tensor(out)
Expand Down