Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't pass derived args to forw pass unless they are references #1207

Merged
merged 1 commit into from
Jan 5, 2025

Conversation

PetroZarytskyi
Copy link
Collaborator

_reverse_forward functions only contain the forward pass, which only affects the derivatives on references/pointers. For other types, there is no point in passing the adjoint. Moreover, doing so is often incorrect because derived arguments are generated for the reverse pass, e.g.

// forward pass
operator_subscript_reverse_forw(&vec, 0, &_d_vec, _r0); // `_r0` is declared later
...
// reverse pass
size_type _r0 = 0UL;
operator_subscript_pullback(&_t2, 0, 1, &_d_vec, &_r0);

In this example, replacing the first occurrence of _r0 with 0 will still be correct.

Copy link
Contributor

github-actions bot commented Jan 3, 2025

clang-tidy review says "All clean, LGTM! 👍"

Copy link

codecov bot commented Jan 3, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.62%. Comparing base (dcb704f) to head (1c57946).
Report is 3 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1207      +/-   ##
==========================================
- Coverage   94.75%   94.62%   -0.13%     
==========================================
  Files          51       51              
  Lines        8883     8877       -6     
==========================================
- Hits         8417     8400      -17     
- Misses        466      477      +11     
Files with missing lines Coverage Δ
lib/Differentiator/ReverseModeVisitor.cpp 95.49% <100.00%> (-0.06%) ⬇️

... and 1 file with indirect coverage changes

Files with missing lines Coverage Δ
lib/Differentiator/ReverseModeVisitor.cpp 95.49% <100.00%> (-0.06%) ⬇️

... and 1 file with indirect coverage changes

Copy link
Owner

@vgvassilev vgvassilev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any chance for a tests? Also can you make the commit message match the PR description?

Copy link
Contributor

github-actions bot commented Jan 4, 2025

clang-tidy review says "All clean, LGTM! 👍"

// CHECK-NEXT: std::vector<double> _t15 = vec;
// CHECK-NEXT: {{.*}}ValueAndAdjoint<double &, double &> _t16 = {{.*}}class_functions::operator_subscript_reverse_forw(&vec, 2, &_d_vec, _r6);
// CHECK-NEXT: {{.*}}ValueAndAdjoint<double &, double &> _t16 = {{.*}}class_functions::operator_subscript_reverse_forw(&vec, 2, &_d_vec, {{0U|0UL|0}});
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Writing FileCheck CHECKs that satisfy all compiler/runtime combinations seems can get very time-consuming. @vgvassilev @PetroZarytskyi What do you think about testing the FileCheck CHECKs for just one compiler/runtime?

Copy link
Owner

@vgvassilev vgvassilev Jan 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, alternatively we can just put auto for the ValueAndAdjoint. I do not think we will need a lot of regex magic after we landed #1171.

Copy link
Contributor

github-actions bot commented Jan 4, 2025

clang-tidy review says "All clean, LGTM! 👍"

@PetroZarytskyi
Copy link
Collaborator Author

Any chance for a tests?

@vgvassilev The coverage decreased because we no longer visit InitListExpr without dfdx (which is, ideally, never supposed to happen). And because of that, InitListExpr is never cloned either. This is not a bad thing itself since cloning is dangerous and should only be applied to simple exprs like literals or decl refs.

@vgvassilev
Copy link
Owner

Any chance for a tests?

@vgvassilev The coverage decreased because we no longer visit InitListExpr without dfdx (which is, ideally, never supposed to happen). And because of that, InitListExpr is never cloned either. This is not a bad thing itself since cloning is dangerous and should only be applied to simple exprs like literals or decl refs.

Understood I can merge.

…inters.

``_reverse_forward`` functions only contain the forward pass, which only affects the derivatives on references/pointers. For other types, there is no point in passing the adjoint. Moreover, doing so is often incorrect because derived arguments are generated for the reverse pass, e.g.
```
// forward pass
operator_subscript_reverse_forw(&vec, 0, &_d_vec, _r0); // `_r0` is declared later
...
// reverse pass
size_type _r0 = 0UL;
operator_subscript_pullback(&_t2, 0, 1, &_d_vec, &_r0);
```
In this example, replacing the first occurrence of _r0 with 0 will still be correct.
Copy link
Contributor

github-actions bot commented Jan 5, 2025

clang-tidy review says "All clean, LGTM! 👍"

@vgvassilev vgvassilev merged commit 1fcd683 into vgvassilev:master Jan 5, 2025
89 of 90 checks passed
@PetroZarytskyi PetroZarytskyi deleted the forw-arg branch January 6, 2025 12:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants