-
Notifications
You must be signed in to change notification settings - Fork 559
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Expose jitter #2136
Expose jitter #2136
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm other than the lint errors and the import question!
Thanks for this @hughsalimbeni ! If you wouldn't mind also making these changes to the other variational strategies that would be great! |
Co-authored-by: Geoff Pleiss <[email protected]>
Co-authored-by: Geoff Pleiss <[email protected]>
In the last commit I found one more in the grid interpolation strategy, but this time it breaks with anything smaller than 1e-3, so I've left explicit for now. The default is now changed from 1e-3 to 1e-6 (or 1e-8 for float64), which is quite a large change. Might this cause problems for some users? |
I imagine this should be okay - we had 1e-3 or 1e-4 in our ExactGP code until we moved to the context manager, and (AFAIK) it hasn't introduced too many numerical issues. We can always tighten this (or make a separate setting for variational models) if we see trouble down the line). Thanks @hughsalimbeni ! |
Head branch was pushed to by a user without write access
Odd that the |
Interesting also that it's failing against pytorch master only, but not pytorch's current stable release. Maybe 1e-6 is problematically small for variational GPs in fp32 and we should make a |
I'm going to add that setting, and then merge. |
One small issue is that the previous behaviour had both
1e-3
and1e-4
added in various places. I've only found one test that breaks with 1e-4 for both, so I've set this one to 1e-3 via the new exposedjitter_val
arg.Happy to do the other variational strategies, and move the defaults to the settings.