Regularization & standardization #111
Conversation
mhowlan3
commented
Mar 23, 2021
mhowlan3
commented
- Log-posterior calculation in MCMC
- SVD regularization via truncation
- Re-normalization (standardization) of output space
| full_cov = Diagonal(gvar) | ||
| eigs = eigvals(full_cov) | ||
| log_gpfidelity = -FT(0.5) * sum(log.(eigs)) | ||
| # Combine got log_rho |
There was a problem hiding this comment.
In this version of MCMC, we are actually assuming that gvar is a vector of floats, I'm thinking maybe we should relax this to take in matrices. Perhaps the easiest way is to rewrite the current mcmc_sample! function to take in matrix input gcov and add this extra function
function mcmc_sample!(mcmc::MCMC{FT}, g::Vector{FT}, gvar::Vector{FT}) where {FT}
return mcmc_sample!(mcmc, g, Diagonal(gvar))
endThere was a problem hiding this comment.
I've just checked that if eigvals receives a Diagonal type matrix it really does just take the diagonal as the eigenvalues. and so is a nanosecond timed computation still, and so the above should still take advantage of the structure
There was a problem hiding this comment.
Great! I also wrote it this way in the event that full_cov becomes a full matrix in the future, rather than just diagonal.
There was a problem hiding this comment.
I am not sure you have pushed this yet? I can't see it it
src/GaussianProcessEmulator.jl
Outdated
| @@ -66,11 +76,13 @@ struct GaussianProcess{FT<:AbstractFloat, GPM} | |||
| "the Gaussian Process (GP) Regression model(s) that are fitted to the given input-data pairs" | |||
| models::Vector | |||
| "the singular value decomposition of obs_noise_cov, such that obs_noise_cov = decomposition.U * Diagonal(decomposition.S) * decomposition.Vt." | |||
| decomposition::Union{SVD, Nothing} | |||
| decomposition::Union{SVD, decomp_struct, Nothing} | |||
There was a problem hiding this comment.
It's fine to create the decomposition structure here,
might be nice to save everything in you decomp_struct format, then you can use Union{decomp_struct, Nothing} as the type.
You just need to add a constructor for decomp_struct for the non-truncated case, (something like this?)
decomp_struct(svd::SVD) = decomp_struct(svd.V, svd.Vt, svd.S, size(svd.V)[0])and then create the structs
decomposition = decomp_struct(svd(obs_noise_cov))
There was a problem hiding this comment.
Thanks very much Mike!
- If you agree about the decomposition type, could you change that, should be a quick change
- Check that you wish to keep all the
printlnstatements in the source files, and also there are a couple of# TO DOcomments that could be removed in the source files (check the diff) - Codecov report (if you go through the testing) shows a +1.43% increase
Otherwise LGTM!
Codecov Report
@@ Coverage Diff @@
## master #111 +/- ##
==========================================
- Coverage 88.98% 80.90% -8.09%
==========================================
Files 4 4
Lines 336 377 +41
==========================================
+ Hits 299 305 +6
- Misses 37 72 +35
Continue to review full report at Codecov.
|
|
bors r+ |
111: Regularization & standardization r=mhowlan3 a=mhowlan3 1. Log-posterior calculation in MCMC 2. SVD regularization via truncation 3. Re-normalization (standardization) of output space Co-authored-by: mhowlan3 <[email protected]>
|
Timed out. |
|
bors r+ |
|
Build succeeded: |
