Add support for reading designmatrix #8800

larsevj · 2024-09-25T14:03:46Z

Issue
Resolves #8902

Supports reading excel files containing design matrix with ert parameters into pandas dataframes.

PR title captures the intent of the changes, and is fitting for release notes.
Added appropriate release note label
Commit history is consistent and clean, in line with the contribution guidelines.
Make sure unit tests pass locally after every commit (git rebase -i main --exec 'pytest tests/ert/unit_tests -n logical -m "not integration_test"')

When applicable

When there are user facing changes: Updated documentation
New behavior or changes to existing untested code: Ensured that unit tests are added (See Ground Rules).
Large PR: Prepare changes in small commits for more convenient review
Bug fix: Add regression test for the bug
Bug fix: Create Backport PR to latest release

codecov-commenter · 2024-10-03T11:38:37Z

Codecov Report

Attention: Patch coverage is 96.29630% with 3 lines in your changes missing coverage. Please review.

Project coverage is 91.70%. Comparing base (fc6244f) to head (87fa0ff).
Report is 9 commits behind head on main.

Files with missing lines	Patch %	Lines
src/ert/config/design_matrix.py	96.05%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8800      +/-   ##
==========================================
+ Coverage   91.65%   91.70%   +0.04%     
==========================================
  Files         343      343              
  Lines       21168    21231      +63     
==========================================
+ Hits        19402    19470      +68     
+ Misses       1766     1761       -5

Flag	Coverage Δ
cli-tests	`39.78% <27.16%> (-0.05%)`	⬇️
gui-tests	`73.66% <27.16%> (-0.17%)`	⬇️
performance-tests	`50.32% <27.16%> (-0.08%)`	⬇️
unit-tests	`80.38% <96.29%> (+0.07%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

larsevj · 2024-10-03T12:53:39Z

A lot of tests can probably be ported over from Semeio.

jonathan-eq · 2024-10-04T10:49:08Z

src/ert/config/gen_kw_config.py

@@ -90,6 +88,9 @@ def __post_init__(self) -> None:
                )
        self._validate()

+    def __contains__(self, item: str) -> bool:
+        return item in [v.name for v in self.transform_function_definitions]


Can do this lazily?

Suggested change

return item in [v.name for v in self.transform_function_definitions]

return any(item in v.name for v in self.transform_function_definitions)

Not sure if that does the same thing, but will see if there is a better way for this?

src/ert/config/design_matrix.py

tests/ert/unit_tests/sensitivity_analysis/test_design_matrix.py

xjules

Very nice PR @larsevj ! 🎉 🚀
Could you squash the commits since I think you can't really separate it to a clever chunks.

…IX config entry

- Adds functionality to the DesignMatrix dataclass for reading design matrix excel files into pandas dataframes, akin to design2params in semeio.

larsevj force-pushed the add_support_for_reading_designmatrix branch 3 times, most recently from 6fe00b5 to 3165420 Compare September 26, 2024 13:09

larsevj force-pushed the add_support_for_reading_designmatrix branch 2 times, most recently from 1d0ec6d to b45b9f6 Compare October 3, 2024 11:23

larsevj force-pushed the add_support_for_reading_designmatrix branch from b45b9f6 to 92ddf3a Compare October 3, 2024 12:51

jonathan-eq self-requested a review October 4, 2024 10:12

jonathan-eq reviewed Oct 4, 2024

View reviewed changes

larsevj force-pushed the add_support_for_reading_designmatrix branch 3 times, most recently from 4dc66f7 to 7e0671a Compare October 8, 2024 09:05