Standardize on Julia 1.6.2 #127
Conversation
120: Modular emulator interface r=odunbar a=odunbar
## Purpose
Reduces the current Emulator interface dependence of Gaussian Processes. Now the GP can be swapped for another statistical emulator.
## In the PR
- [x] New general `Emulator` class. This handles all the data manipulation e.g. Normalization, Standardization, Decorrelation
- [x] General interface functions for Emulator, `optimize_hyperparameters!`, `predict`
- [x] New `MachineLearningTool` type,
- [x] Moved the Gaussian Processes into a `GaussianProcess <: MachineLearningTool` class
- [x] Example (e.g. `plot_GP`) to demonstrate the new interface
- [x] Unit tests
- [x] New doc strings.
## Additional change
Seems to be ongoing issues with unit testing Julia 1.5.4, so I have updated the Manifest, Docs.yml and Test.yml to Julia 1.6.X
## Changes to user experience:
Ingredients:
```julia
gppackage = GPJL()
pred_type = YType()
GPkernel = ...
iopairs = PairedDataContainer(x_data,y_data)
```
### Old interface
Set up a `GaussianProcessEmulator` object
```julia
gp = GaussianProcess(
iopairs,
gppackage;
GPkernel=GPkernel,
obs_noise_cov=nothing,
normalized=false,
noise_learn=true,
truncate_svd=1.0,
standardize=false,
prediction_type=pred_type,
norm_factor=nothing)
```
Then predict with it.
```julia
μ, σ² = GaussianProcessEmulator.predict(gp, new_inputs)
```
It is short, but it inherently is stuck to the Gaussian process framework. It also hides e.g. the training away, and we may wish to have this more open. The script below is more general, separating out which parameters are related to data processing and which relate to the specific ML tool.
### New interface
Setup a `GaussianProcess<:MachineLearningTool` object
```julia
gp = GaussianProcess(
gppackage;
kernel=GPkernel,
noise_learn=true,
prediction_type=pred_type)
```
and then create the general emulator type using `gp`
```julia
em = Emulator(
gp,
iopairs,
obs_noise_cov=nothing,
normalize_inputs=false,
standardize_outputs=false,
truncate_svd=1.0)
```
Train and predict
```julia
Emulators.optimize_hyperparameters!(em)
μ, σ² = Emulators.predict(em, new_inputs)
```
### Adding a new `MachineLearningTool`
Include a new file `NewTool.jl` at the top of `Emulator.jl`
In this file define:
1. `struct NewTool <: MachineLearningTool` with constructor `NewTool(...)` to hold ML parameters and models
2. `function build_models!(NewTool,iopairs)` to build and store ML models. Called in Emulator constructor
3. `function optimize_hyperparameters!(NewTool)`to train the stored ML models. Called by method of same name in Emulator
4. `function predict(NewTool,new_inputs)` to predict with stored ML models Called by method of same name in Emulator
Co-authored-by: odunbar <[email protected]>
|
bors try |
tryBuild failed: |
|
it seems like the examples are broken? |
Yeah, that's what I've got at this point: this PR is up-to-date with For what it's worth, unit tests pass on |
Done because noise is explicitly added to the GP kernel when it's created. This is needed to reproduce existing behavior in /master.
| else | ||
| println(truth.mean) | ||
| end | ||
| println(truth.mean) # same, regardless of norm_factor |
There was a problem hiding this comment.
We shouldn't have needed to rescale truth.mean, since that's what the emulator is emulating, and in the new code y_mean is correctly scaled by reverse_standardize. Verified that vec(y_mean) ≈ truth.mean in the new code, which means this was being done incorrectly (but consistently) on the existing code in /master.
|
Output of examples now either reproduce @jakebolewski, we're ready to rerun buildkite when you get a sec. Thanks! |
|
bors try |
tryBuild failed: |
|
CalibrateEmulateSample.jl/.buildkite/pipeline.yml Lines 56 to 68 in ce6db0e need to reflect the new example path (also for the artifacts) |
Argh, can't believe I missed that! Thanks for catching that mistake, @jakebolewski ! |
|
bors r+ |
|
Build failed: |
|
It looks like the load path needs to be adjusted in the buildkite file |
|
Thanks again @jakebolewski -- this is fixed in new PR #128, since I can't re-open this one. Verified that the GaussianProcess examples now run when the exact commands in pipeline.yml are invoked from the repo dir (previously was running from the examples' directory, with the CES module previously loaded in the REPL, which I'm guessing is why I didn't hit the error earlier.) |
This is a revision of PR #126, which ran into a new problem (missing environment module) when attempting to fix #125 (Manifest.toml incompatibility between julia 1.5 and 1.6). See most recent comment on that PR.
Current PR pins the buildkite julia version at 1.6.2, for which a module currently exists.
The
Manifests in this PR were generated under 1.6.5, and I haven't been able to verify that they're compatible with 1.6.2 (julialang.org didn't retain 1.6.2, and the conda package for 1.6.2 appears to be broken on osx).