Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
closes #619
This makes for a nice canonical way to decide if something should be pulled back.
Since even complicated structures like:
work right.
So this is better than just checking top-level type.
It is much cheaper to check if something is zero, even if it involved elementwise work than the vast majority of pullbacks so i think a case can be made that this should always been done by the AD. Or at least the AD should have special cases for things to treat as as always nonzero like dense numerical mastrixes
<:Array{<:Real}
(which are also things that rules support).This should go well with FluxML/Zygote.jl#1389