Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate that DESIGN_MATRIX is correct upon initialization #8686

Open
Tracked by #8688
xjules opened this issue Sep 12, 2024 · 3 comments
Open
Tracked by #8688

Validate that DESIGN_MATRIX is correct upon initialization #8686

xjules opened this issue Sep 12, 2024 · 3 comments

Comments

@xjules
Copy link
Contributor

xjules commented Sep 12, 2024

The validation should include checking whether the design sheet part handles NUM_REALIZATIONS correctly, which means whether the all the design values combined with default sheet will handle the total number of realizations.

  • REAL column (if provided) should contain iens; which yield active realizations and thus filling the active realizations edit box.
  • in default sheet if the parameter is already present in the design sheet the default sheet entry is ignored otherwise it is appended as a new column with a singular default value.
  • make sure that the parameter names are unique

Blocked by: #8902

@xjules xjules changed the title Validate that DESIGN_MATRIX is correct upon initialization (num reals vs. num params vs defaults) Validate that DESIGN_MATRIX is correct upon initialization Sep 12, 2024
@xjules xjules moved this to Todo in SCOUT Sep 12, 2024
@xjules xjules self-assigned this Sep 25, 2024
@larsevj
Copy link
Contributor

larsevj commented Sep 26, 2024

Currently design2params will fail if you run more realizations than entries in design matrix. Maybe we should do the same, just ignore NUM_REALIZATIONS if you choose design matrix and use number of realizations specified in the design matrix.
For instance if you only specify realization 1,4,7 in the design matrix, than it is probably expected that only realizations 1,4,7 are actually run?

@xjules
Copy link
Contributor Author

xjules commented Sep 30, 2024

Currently design2params will fail if you run more realizations than entries in design matrix. Maybe we should do the same, just ignore NUM_REALIZATIONS if you choose design matrix and use number of realizations specified in the design matrix. For instance if you only specify realization 1,4,7 in the design matrix, than it is probably expected that only realizations 1,4,7 are actually run?

This makes sense for ensemble_experiment, but not sure if we are to use DESIGN_MATRIX in the update step. Let's focus only for the ensemble experiment for now. If we would choose a subset (active realizations) from the list of realizations in the design matrix, why would this not work?

@xjules xjules added the blocked label Oct 7, 2024
@xjules xjules moved this from In Progress to Todo in SCOUT Oct 7, 2024
@larsevj
Copy link
Contributor

larsevj commented Oct 11, 2024

Some of the points of this issue has been covered but are still a lot of validation issues, some of them are:

  • Disallow spaces in parameter names
  • Decide if we will allow parameter names to be numbers or not
  • Handle empty input properly or misconfigured input, might rely on pandas for this. But maybe we should check what kind of error messages we receive and decide if we want to show these or display our own errors.
  • Find out what errors are dependent on each others and what are not, so that we can validate as much as possible before raising error?

@xjules xjules removed the blocked label Oct 14, 2024
@xjules xjules moved this from In Progress to Todo in SCOUT Oct 15, 2024
@xjules xjules removed their assignment Oct 15, 2024
@xjules xjules moved this from Todo to In Progress in SCOUT Dec 13, 2024
@xjules xjules self-assigned this Dec 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: In Progress
Development

No branches or pull requests

3 participants