A Jacobian which understands Params #414
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
XRef: I implemented one in #235 as well but I didn't do anything on Params |
|
Oh I completely missed that, sorry. Perhaps something to handle Params should just be bolted onto that. |
|
Ah no, sorry, it's now called |
|
Thanks! It's working perfectly now! |
Closed
Merged
bors bot
added a commit
that referenced
this pull request
Feb 2, 2021
890: Add `jacobian`, at last? r=CarloLucibello a=mcabbott This adds a Jacobian function. Compared to #747 this one: * has tests * accepts multiple arguments * shouldn't fail if `back` returns `nothing` * inserts `vec` a few more places * has a method which accepts implicit `Params`, and returns `Grads`. (This was the only part I actually needed in real life.) * now works on the GPU too! Compared to #414 this one: * always inserts `vec`, never makes higher-dimensional arrays * runs on current Zygote * has tests. Compared to #235 this one: * doesn't try to provide numerical jacobian * doesn't alter testing infrastructure * doesn't provide `jacobian!`, nor any code for structured matrices. This does not address #564's concerns about functions which return a tuple of arrays. Only functions returning an array (or a scalar) are permitted. Similar considerations might give sensible jacobians when the argument of a function is a tuple, or some other struct, but for now these are handled by putting up a giant warning sign. Nothing in the file `utils.jl` seems to have any tests at all. So I also added tests for `hessian`. And, while I was there, made `hessian` actually accept a real number like its docstring promises. (Hence this closes #891.) And, made a version that is reverse-over-reverse, using this `jacobian`, which works less well of course but may as well exist to test things. (See for example #865.) Ideally there would be a pure-Zygote version using its own forward mode, but I didn't write that. Fixes #51, fixes #98, fixes #413. Closes #747. Co-authored-by: Michael Abbott <me@escbook> Co-authored-by: Michael Abbott <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #413, and #51, and half of #98.
Questions:
Should
jacobian(f, ps::Params)return something other than anIdDict()?Should
matrix=truebe the default? Needs to be added to Params method.Needs tests obviously.