Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add declarations for variables reused in the reverse pass #688

Closed
wants to merge 1 commit into from

Conversation

PetroZarytskyi
Copy link
Collaborator

@PetroZarytskyi PetroZarytskyi commented Dec 22, 2023

The problem this PR attempts to fix is described in #659 (which in turn caused #681). Also, this PR introduces placeholders to DelayedGlobalStoreAndRef to rewrite expressions after they are added to the current block if there is no point in storing them. This is done to avoid cloning while handling multiplication differentiation.

Fixes #659. Fixes #681.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

lib/Differentiator/ReverseModeVisitor.cpp Outdated Show resolved Hide resolved
lib/Differentiator/ReverseModeVisitor.cpp Outdated Show resolved Hide resolved
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

lib/Differentiator/ReverseModeVisitor.cpp Outdated Show resolved Hide resolved
@parth-07
Copy link
Collaborator

@PetroZarytskyi Can you please describe the solution in the commit message / pull request description?

@PetroZarytskyi PetroZarytskyi force-pushed the local-vars branch 3 times, most recently from b54ec5f to dea82f4 Compare January 6, 2024 11:52
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

lib/Differentiator/MultiplexExternalRMVSource.cpp Outdated Show resolved Hide resolved
lib/Differentiator/ReverseModeVisitor.cpp Outdated Show resolved Hide resolved
lib/Differentiator/ReverseModeVisitor.cpp Outdated Show resolved Hide resolved
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

lib/Differentiator/ReverseModeVisitor.cpp Outdated Show resolved Hide resolved
lib/Differentiator/ReverseModeVisitor.cpp Outdated Show resolved Hide resolved
lib/Differentiator/ReverseModeVisitor.cpp Outdated Show resolved Hide resolved
lib/Differentiator/ReverseModeVisitor.cpp Outdated Show resolved Hide resolved
lib/Differentiator/ReverseModeVisitor.cpp Outdated Show resolved Hide resolved
@PetroZarytskyi PetroZarytskyi force-pushed the local-vars branch 3 times, most recently from c2aa94c to 17d22e6 Compare January 11, 2024 15:41
@vgvassilev vgvassilev marked this pull request as ready for review January 12, 2024 15:34
@vgvassilev vgvassilev requested a review from parth-07 January 12, 2024 15:34
Copy link
Owner

@vgvassilev vgvassilev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is some partial review. Could you go over your FIXME notes and try to address them as it seems most of them need to be addressed within this PR.

lib/Differentiator/CladUtils.cpp Show resolved Hide resolved
include/clad/Differentiator/ReverseModeVisitor.h Outdated Show resolved Hide resolved
include/clad/Differentiator/ReverseModeVisitor.h Outdated Show resolved Hide resolved
lib/Differentiator/ReverseModeVisitor.cpp Outdated Show resolved Hide resolved
lib/Differentiator/ReverseModeVisitor.cpp Outdated Show resolved Hide resolved
lib/Differentiator/ReverseModeVisitor.cpp Outdated Show resolved Hide resolved
lib/Differentiator/ReverseModeVisitor.cpp Outdated Show resolved Hide resolved
lib/Differentiator/ReverseModeVisitor.cpp Outdated Show resolved Hide resolved
cond = Visit(FS->getCond());
const auto* IDRE = dyn_cast<DeclRefExpr>(FS->getInc());
bool IDRE = isa<DeclRefExpr>(FS->getInc());
/// FIXME: Why do we need to differentiate DeclRefExpr twice?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which DeclRefExpr are you referring here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I get, inc is supposed to be the original increment (it's always differentiated later). Why is it differentiated specifically when it is a DeclRefExpr?

StmtDiff IdxDiff = Visit(Indices[i]);
clonedIndices[i] = Clone(IdxDiff.getExpr());
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this change necessary for this pull request? If not, then this should be fixed in a separate pull-request.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it's necessary to replace getExpr with getRevSweepAsExpr for reverseIndices. I could move Clone to where they were previously if that matters.

//CHECK-NEXT: clad::push(_t1, t);
//CHECK-NEXT: t *= x;
//CHECK-NEXT: {
//CHECK-NEXT: int i;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it necessary to declare the loop init statement outside the loop?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue here is that we want to store the last value of the loop init. So we need to access i once right after the last condition. But we can't do that outside the loop since the variable is not declared there. My solution doesn't too look nice though so if you have a better idea feel free to share it.

@PetroZarytskyi PetroZarytskyi force-pushed the local-vars branch 3 times, most recently from 9d152f6 to c6961bf Compare January 15, 2024 10:12
Copy link

codecov bot commented Jan 15, 2024

Codecov Report

Attention: 6 lines in your changes are missing coverage. Please review.

Comparison is base (d8ef790) 94.54% compared to head (1661a22) 94.64%.
Report is 2 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #688      +/-   ##
==========================================
+ Coverage   94.54%   94.64%   +0.09%     
==========================================
  Files          49       49              
  Lines        7189     7281      +92     
==========================================
+ Hits         6797     6891      +94     
+ Misses        392      390       -2     
Files Coverage Δ
include/clad/Differentiator/ErrorEstimator.h 100.00% <ø> (ø)
...e/clad/Differentiator/MultiplexExternalRMVSource.h 100.00% <ø> (ø)
include/clad/Differentiator/VisitorBase.h 100.00% <ø> (ø)
lib/Differentiator/CladUtils.cpp 97.08% <100.00%> (ø)
lib/Differentiator/ErrorEstimator.cpp 98.51% <100.00%> (+<0.01%) ⬆️
lib/Differentiator/MultiplexExternalRMVSource.cpp 90.81% <100.00%> (+0.29%) ⬆️
lib/Differentiator/VisitorBase.cpp 98.05% <100.00%> (+<0.01%) ⬆️
include/clad/Differentiator/ExternalRMVSource.h 23.07% <0.00%> (-0.61%) ⬇️
lib/Differentiator/ReverseModeVisitor.cpp 96.35% <99.60%> (+0.26%) ⬆️
include/clad/Differentiator/ReverseModeVisitor.h 95.74% <83.33%> (-1.93%) ⬇️
... and 1 more

... and 1 file with indirect coverage changes

Files Coverage Δ
include/clad/Differentiator/ErrorEstimator.h 100.00% <ø> (ø)
...e/clad/Differentiator/MultiplexExternalRMVSource.h 100.00% <ø> (ø)
include/clad/Differentiator/VisitorBase.h 100.00% <ø> (ø)
lib/Differentiator/CladUtils.cpp 97.08% <100.00%> (ø)
lib/Differentiator/ErrorEstimator.cpp 98.51% <100.00%> (+<0.01%) ⬆️
lib/Differentiator/MultiplexExternalRMVSource.cpp 90.81% <100.00%> (+0.29%) ⬆️
lib/Differentiator/VisitorBase.cpp 98.05% <100.00%> (+<0.01%) ⬆️
include/clad/Differentiator/ExternalRMVSource.h 23.07% <0.00%> (-0.61%) ⬇️
lib/Differentiator/ReverseModeVisitor.cpp 96.35% <99.60%> (+0.26%) ⬆️
include/clad/Differentiator/ReverseModeVisitor.h 95.74% <83.33%> (-1.93%) ⬇️
... and 1 more

... and 1 file with indirect coverage changes

include/clad/Differentiator/ReverseModeVisitor.h Outdated Show resolved Hide resolved
lib/Differentiator/ReverseModeVisitor.cpp Outdated Show resolved Hide resolved
lib/Differentiator/ReverseModeVisitor.cpp Outdated Show resolved Hide resolved
lib/Differentiator/ReverseModeVisitor.cpp Show resolved Hide resolved
lib/Differentiator/ReverseModeVisitor.cpp Outdated Show resolved Hide resolved
lib/Differentiator/ReverseModeVisitor.cpp Outdated Show resolved Hide resolved
lib/Differentiator/ReverseModeVisitor.cpp Outdated Show resolved Hide resolved
include/clad/Differentiator/ReverseModeVisitor.h Outdated Show resolved Hide resolved
lib/Differentiator/ReverseModeVisitor.cpp Show resolved Hide resolved
// If the declaration is not located in the function global scope,
// we have to redeclare it in the reverse sweep since it won't be
// visible otherwise.
// FIXME: Is there a better way to determine non-function-global scope
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, there must be a better way for this. Have you tried VD->getDeclContext() != m_Function?
Ideally, function global scope variables should have the primal FunctionDecl as their DeclContext

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how DeclContext works exactly but getDeclContext seems to return m_Function for variables declared inside loops etc.

lib/Differentiator/ReverseModeVisitor.cpp Show resolved Hide resolved
@parth-07
Copy link
Collaborator

The produced gradient of the below code has compile-time error in the local variable redeclaration part.

double fn(double u, double v) {
    double r;
    {
        double res;
        double v_copy = v;
        double &v_ref = v_copy;
        res = v_ref;
        r = res;
    }
    return r;
}

The local variable redeclaration in the reverse sweep is as follows:

double &v_ref = v_copy;
double v_copy = _t3;
double res = _t2;

v_ref should be declared after v_copy.

m_Blocks.pop_back();
else
EmitReverseSweepDeclarations();
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a test case covering this branch?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, EndBlockWithoutCreatingCS is only used once in the reverse direction. There is no way to call it in the forward direction just with tests.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that dead code then?

@vgvassilev
Copy link
Owner

@PetroZarytskyi, is that PR superseded by #738? Can we close it?

@PetroZarytskyi
Copy link
Collaborator Author

@PetroZarytskyi, is that PR superseded by #738? Can we close it?

I think the solution implemented in #738 is better. We should probably close this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants