Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding constrained sensing capabilities to the SPSL post processor. Added a test_SPSL_constrained xml for testing. #2346

Open
wants to merge 57 commits into
base: devel
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
5055f61
starting classification
Jimmy-INL Nov 5, 2023
32f6a5c
adding classification data
Jimmy-INL Nov 6, 2023
8a1b5d5
hard coding the labels for now till we decide on naming
Jimmy-INL Feb 20, 2024
66b4adf
adding the state variable and remove the hard coding
Jimmy-INL Feb 23, 2024
226b5a8
adding test and golded files
Jimmy-INL Feb 23, 2024
53c72e3
Adding modified SPSL with constraints
Jun 4, 2024
3c19e83
Adding test file for constraints SPSL
Jun 4, 2024
9c435e0
Merge branch 'SPSL_add_constraints' of https://github.com/niharika299…
Jul 29, 2024
a092359
Adding path the pysensors branch for constraints in dependencies and …
Jul 29, 2024
047fdf1
deleting trailing whitespaces in the file SparseSensing.py
niharika2999 Jul 30, 2024
357a16a
Removing git+ from dependencies as the library handler adds git+
niharika2999 Jul 30, 2024
bd72312
Fixing constrainedRegion being None error
niharika2999 Jul 30, 2024
4dbfa69
Changing variable name target to measuredState in the workshop SPSL test
niharika2999 Jul 30, 2024
4a71208
Fixing constrainedRegions Typo
niharika2999 Jul 30, 2024
5a55ae4
Fixing missing else statement for ConstrainedRegions
niharika2999 Jul 30, 2024
207b0cf
Fixing error by including _ConstrainedRegionsType= None
niharika2999 Aug 1, 2024
b59e4d0
Adding the test from workshop with target changed to measuredState
niharika2999 Aug 1, 2024
d2dc13b
Changing target to measuredState in the workshop inputs folder
niharika2999 Aug 1, 2024
a288140
Updating documentation for the sparse sensing postprocessor
niharika2999 Aug 2, 2024
fb43bec
Fixing \_n issues for max_n and exact_n in the documentation
niharika2999 Aug 2, 2024
26284ee
Adding \_ to the documentation inside xml node and removing the typo …
niharika2999 Aug 2, 2024
3f7b970
Changing xy_coords in the documentation
niharika2999 Aug 5, 2024
60517e5
Adding doc, changes to SparseSensing.py and tests for each shape
niharika2999 Aug 5, 2024
ef79d99
Fixing .csv to ./csv in the tests
niharika2999 Aug 5, 2024
a4a1159
Adding the folders that contain csv data in tests for the constraints
niharika2999 Aug 5, 2024
b6d4a9d
Changes to xml's
niharika2999 Aug 5, 2024
aa0a10a
Fixing xml and adding missed line test
niharika2999 Aug 5, 2024
b31c4fa
Changes to the SparseSensing postprocessor
niharika2999 Aug 6, 2024
44b71f7
Changes to the SparseSensing postprocessor
niharika2999 Aug 6, 2024
bf3d244
Trying to remove whitespace
niharika2999 Aug 6, 2024
d09da7e
Deleting trailing whitespace
niharika2999 Aug 6, 2024
dbb792d
Fixed Sparsesesning postprocessor
niharika2999 Aug 15, 2024
c6d40b4
Adding working SparseSensing
niharika2999 Aug 15, 2024
4618f3d
Almost finished SparseSensing.py and gold files for circle, ellipse a…
niharika2999 Aug 16, 2024
0377df7
Adding files for gold folder in tests
niharika2999 Aug 19, 2024
cfba5ed
Changing cloudpickle dependency from2.2 to 3.0
niharika2999 Aug 19, 2024
8155b90
adding circle, polygon, ellipse to gold in tests
Sep 16, 2024
c06a2bb
Merge branch 'devel' into SPSL_add_constraints
niharika2999 Sep 16, 2024
9eaf38b
Trying to fix errors for circle and ellipse constraints tests
Sep 16, 2024
254895a
Merge branch 'SPSL_add_constraints' of https://github.com/niharika299…
Sep 16, 2024
bf98b39
Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…
niharika2999 Sep 17, 2024
03428a5
Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…
niharika2999 Sep 17, 2024
4975ad8
Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…
niharika2999 Sep 17, 2024
7e12525
Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…
niharika2999 Sep 17, 2024
ad67ac2
Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…
niharika2999 Sep 17, 2024
995c889
Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…
niharika2999 Sep 17, 2024
f143170
Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…
niharika2999 Sep 17, 2024
d66eae9
Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…
niharika2999 Sep 17, 2024
21d686b
Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…
niharika2999 Sep 17, 2024
9e5835e
Changing working directory for the tests
Sep 17, 2024
0b1b272
Merge branch 'SPSL_add_constraints' of https://github.com/niharika299…
Sep 17, 2024
307dabe
Renaming folders for tests
Sep 17, 2024
6bcca52
Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…
niharika2999 Sep 17, 2024
a5b9be5
Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…
niharika2999 Sep 17, 2024
676fd20
Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…
niharika2999 Sep 17, 2024
99066b5
Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…
niharika2999 Sep 17, 2024
e3b6a09
Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…
niharika2999 Sep 17, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion dependencies.xml
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ Note all install methods after "main" take
<!-- <ete3 optional='True'/> -->
<statsforecast/>
<pywavelets optional='True'>1.2</pywavelets>
<python-sensors source="pip"/>
<python-sensors source="pip" repo='https://github.com/Jimmy-INL/pysensors.git@niharika2999-Ni_converting_functional-constraints_class'></python-sensors>
<numdifftools source="pip">0.9</numdifftools>
<fmpy optional='True'/>
<xmlschema source="pip"/>
Expand Down
20 changes: 10 additions & 10 deletions doc/workshop/SparseSensing/exercises/testSPSLOptiTwist.xml
Original file line number Diff line number Diff line change
Expand Up @@ -18,24 +18,24 @@
</RunInfo>

<Files>
<Input name="twistFile" type=""></Input><!-- Provide the csv file that contains the names of the samples files-->
<Input name="refDO" type=""></Input><!-- Provide the csv of the case onwhich you want to show the sensors i.e., maximum initial power (350 W) and maximum initial Temperature-->
<Input name="twistFile" type="">../../data/Perturbations.csv</Input><!-- Provide the csv file that contains the names of the samples files-->
<Input name="refDO" type="">../../data/350_330.csv</Input><!-- Provide the csv of the case onwhich you want to show the sensors i.e., maximum initial power (350 W) and maximum initial Temperature-->
</Files>

<Steps>
<IOStep name="LoadCSV">
<Input class="Files" type="">twistFile</Input>
<Input class="Files" type=""></Input><!--Fill in-->
<Input class="Files" type="">refDO</Input><!--Fill in-->
<Output class="DataObjects" type="HistorySet">TwistDO</Output>
<Output class="DataObjects" type="PointSet">myDO</Output>
</IOStep>
<PostProcess name="mySPpp">
<Input class="DataObjects" type="HistorySet">TwistDO</Input>
<Model class="Models" type="PostProcessor"></Model><!--Fill in-->
<Model class="Models" type="PostProcessor">mySPSL</Model><!--Fill in-->
<Output class="DataObjects" type="DataSet">outPP</Output>
</PostProcess>
<IOStep name="print">
<Input class="DataObjects" type="DataSet">outPP</Input>
<Input class="DataObjects" type="DataSet">outPP</Input>cd
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is cd a typo?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes! Thank you for catching that. I have removed cd.

<Input class="DataObjects" type="PointSet">myDO</Input>
<Input class="DataObjects" type="HostorySet">TwistDO</Input>
<Output class="OutStreams" type="Print">outPP</Output>
Expand All @@ -45,12 +45,12 @@

<Models>
<PostProcessor name="mySPSL" subType="SparseSensing" verbosity="debug">
<Goal subType=""> <!--Fill in-->
<features></features><!--Fill in att variable names needed for training from the CSV-->
<target>Temperature (K)</target>
<Goal subType="reconstruction"> <!--Fill in-->
<features>X (m), Y (m), Temperature (K)</features><!--Fill in att variable names needed for training from the CSV-->
<measuredState>Temperature (K)</measuredState>
<basis>SVD</basis>
<nModes></nModes> <!--Try different modes-->
<nSensors></nSensors> <!--Try different number of sensors-->
<nModes>4</nModes> <!--Try different modes-->
<nSensors>4</nSensors> <!--Try different number of sensors-->
<optimizer>QR</optimizer>
</Goal>
</PostProcessor>
Expand Down
2 changes: 1 addition & 1 deletion doc/workshop/SparseSensing/inputs/testSPSLOptiTwist.xml
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@
<PostProcessor name="mySPSL" subType="SparseSensing" verbosity="debug">
<Goal subType="reconstruction">
<features>X (m),Y (m),Temperature (K)</features>
<target>Temperature (K)</target>
<measuredState>Temperature (K)</measuredState>
<basis>SVD</basis>
<nModes>4</nModes>
<nSensors>4</nSensors>
Expand Down
252 changes: 232 additions & 20 deletions ravenframework/Models/PostProcessors/SparseSensing.py

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Power (W),BCTemperature (K),filename
350,240,350_240.csv
350,270,350_270.csv
350,300,350_300.csv
350,330,350_330.csv
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
sensor,Temperature (K),X (m),Y (m),label
1,241.5795986,0.008141501,-0.098437689,<300 K
2,242.5042687,0.004994706,0.072699441,<300 K
3,240.1987226,0.037176882,0.140548347,<300 K
4,244.1098113,0.003235497,-0.118483818,<300 K
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@
<PostProcessor name="mySPSL" subType="SparseSensing" verbosity="debug">
<Goal subType="reconstruction">
<features>X (m),Y (m),Temperature (K)</features>
<target>Temperature (K)</target>
<measuredState>Temperature (K)</measuredState>
<basis>SVD</basis>
<nModes>4</nModes>
<nSensors>4</nSensors>
Expand All @@ -59,19 +59,19 @@

<DataObjects>
<PointSet name="myDO">
<Output>X (m),Temperature (K),Y (m)</Output>
<Output>X (m), Temperature (K),Y (m)</Output>
</PointSet>
<HistorySet name="TwistDO">
<Input> Power (W), BCTemperature (K)</Input>
<Output>X (m),Temperature (K),Y (m)</Output>
<Output>X (m), Temperature (K), Y (m)</Output>
<options>
<pivotParameter>index</pivotParameter>
</options>
</HistorySet>
<DataSet name="outPP">
<Input>InputPlaceHolder</Input>
<Output>Temperature (K), X (m), Y (m)</Output>
<Index var="sensor">X (m),Y (m),Temperature (K)</Index>
<Index var="sensor">X (m), Y (m), Temperature (K)</Index>
</DataSet>
</DataObjects>

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
<?xml version="1.0" ?>
<Simulation verbosity="debug">
<TestInfo>
<name>tests/framework/Postprocessors/SparseSensing</name>
<author>Mohammad Abdo (@Jimmy-INL)</author>
<created>2023-11-04</created>
<classesTested>Models.Postprocessors.SparseSensing</classesTested>
<description>This test aims to check the ability of the Sparse Sensing Postprocessor
to locate and plot the selected sensors to reconstruct the temperature field of the OPTI-TWIST prototype
when perturbing the heater power.
</description>
</TestInfo>

<RunInfo>
<WorkingDir>classificationOptiTwist</WorkingDir>
<Sequence>LoadCSV, mySPpp, print</Sequence>
<batchSize>1</batchSize>
</RunInfo>

<Files>
<Input name="twistFile" type="">./Perturbations2.csv</Input>
<Input name="refDO" type="">./350_330.csv</Input>
</Files>

<Steps>
<IOStep name="LoadCSV">
<Input class="Files" type="">twistFile</Input>
<Input class="Files" type="">refDO</Input>
<Output class="DataObjects" type="HistorySet">TwistDO</Output>
<Output class="DataObjects" type="PointSet">myDO</Output>
</IOStep>
<PostProcess name="mySPpp">
<Input class="DataObjects" type="HistorySet">TwistDO</Input>
<Model class="Models" type="PostProcessor">mySPSL</Model>
<Output class="DataObjects" type="DataSet">outPP</Output>
</PostProcess>
<IOStep name="print">
<Input class="DataObjects" type="DataSet">outPP</Input>
<Input class="DataObjects" type="PointSet">myDO</Input>
<Input class="DataObjects" type="HostorySet">TwistDO</Input>
<Output class="OutStreams" type="Print">outPP</Output>
<Output class="OutStreams" type="Plot">mySensorPlot</Output>
<Output class="OutStreams" type="Plot">mySensorPlot3D</Output>
</IOStep>
</Steps>

<Models>
<PostProcessor name="mySPSL" subType="SparseSensing" verbosity="debug">
<Goal subType="classification">
<features>X (m),Y (m),Temperature (K), label</features>
<measuredState>Temperature (K)</measuredState>
<labels>label</labels>
<basis>SVD</basis>
<nModes>4</nModes>
<nSensors>4</nSensors>
<classifier>LDA</classifier>
</Goal>
</PostProcessor>
</Models>

<DataObjects>
<PointSet name="myDO">
<Output>X (m),Temperature (K),Y (m), label</Output>
</PointSet>
<HistorySet name="TwistDO">
<Input> Power (W), BCTemperature (K)</Input>
<Output>X (m),Temperature (K),Y (m), label</Output>
<options>
<pivotParameter>index</pivotParameter>
</options>
</HistorySet>
<DataSet name="outPP">
<Input>InputPlaceHolder</Input>
<Output>Temperature (K), X (m), Y (m), label</Output>
<Index var="sensor">X (m),Y (m),Temperature (K), label</Index>
</DataSet>
</DataObjects>

<OutStreams>
<Print name="outPP">
<type>csv</type>
<source>outPP</source>
<what>output</what>
</Print>
<Plot name="mySensorPlot">
<plotSettings>
<plot>
<type>scatter</type>
<x>myDO|Output|X (m)</x>
<y>myDO|Output|Y (m)</y>
<cmap>coolwarm</cmap>
<colorMap>myDO|Output|Temperature (K)</colorMap>
</plot>
<plot>
<type>scatter</type>
<x>outPP|Output|X (m)</x>
<y>outPP|Output|Y (m)</y>
<c>black</c>
<marker>x</marker>
</plot>
<xlabel>x</xlabel>
<ylabel>y</ylabel>
</plotSettings>
<actions>
<how>png</how>
</actions>
</Plot>

<Plot name="mySensorPlot3D">
<plotSettings>
<plot>
<type>scatter</type>
<x>TwistDO|Output|X (m)</x>
<y>TwistDO|Output|Y (m)</y>
<z>TwistDO|Output|Temperature (K)</z>
<cmap>coolwarm</cmap>
<colorMap>TwistDO|Output|Temperature (K)</colorMap>
</plot>
<xlabel>x</xlabel>
<ylabel>y</ylabel>
<zlabel>temperature (K)</zlabel>
</plotSettings>
<actions>
<how>png</how>
</actions>
</Plot>
</OutStreams>

</Simulation>
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
<?xml version="1.0" ?>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file does not seem to be used in the tests yet.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like new tests have been added, so maybe this file can be removed?

<Simulation verbosity="debug">
<TestInfo>
<name>tests/framework/Postprocessors/SparseSensing</name>
<author>Niharika Karnik (@niharika2999), Mohammad Abdo (@Jimmy-INL)</author>
<created>2024-04-01</created>
<classesTested>Models.Postprocessors.SparseSensing</classesTested>
<description>This test aims to check the ability of the Sparse Sensing Postprocessor
to locate and plot the selected sensors to reconstruct the temperature field of the OPTI-TWIST prototype
when perturbing the heater power in the presence of constraints. We assume the region near the heater to have very limited sensing capability and thus accomodate no sensors.
</description>
</TestInfo>

<RunInfo>
<WorkingDir>reconstructionConstraintsOptiTwist</WorkingDir>
<Sequence>LoadCSV, mySPpp, print</Sequence>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@niharika2999 , the reason tests are failing is that the path is inconsistent. For instance, the working directory here is reconstructionConstraintsOptiTwistCircle but in tests it is reconstructionOptiTwistConstraintsCircle. Hope this helps

<batchSize>1</batchSize>
</RunInfo>

<Files>
<Input name="twistFile" type="">./Perturbations2.csv</Input>
<Input name="refDO" type="">./350_330.csv</Input>
</Files>

<Steps>
<IOStep name="LoadCSV">
<Input class="Files" type="">twistFile</Input>
<Input class="Files" type="">refDO</Input>
<Output class="DataObjects" type="HistorySet">TwistDO</Output>
<Output class="DataObjects" type="PointSet">myDO</Output>
</IOStep>
<PostProcess name="mySPpp">
<Input class="DataObjects" type="HistorySet">TwistDO</Input>
<Model class="Models" type="PostProcessor">mySPSL</Model>
<Output class="DataObjects" type="DataSet">outPP</Output>
</PostProcess>
<IOStep name="print">
<Input class="DataObjects" type="DataSet">outPP</Input>
<Input class="DataObjects" type="PointSet">myDO</Input>
<Input class="DataObjects" type="HistorySet">TwistDO</Input>
<Output class="OutStreams" type="Print">outPP</Output>
<Output class="OutStreams" type="Plot">mySensorPlot</Output>
<Output class="OutStreams" type="Plot">mySensorPlot3D</Output>
</IOStep>
</Steps>

<Models>
<PostProcessor name="mySPSL" subType="SparseSensing" verbosity="debug">
<Goal subType="reconstruction">
<features>X (m),Y (m),Temperature (K)</features> ### Try removing features, get X, Y from user
<measuredState>Temperature (K)</measuredState>
<basis>SVD</basis>
<nModes>4</nModes>
<nSensors>4</nSensors>
<nConstSensors>0</nConstSensors>
<constraintOption>"exact_n"</constraintOption>
<ConstrainedRegions type = "circle">
<center_x></center_x> #center coordinates instead (x,y) (r, \theta)
<center_y></center_y>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments in xml are done using <!--your comment-->
center_x and center_y seems not to be recognized

<loc>'in'</loc>
<radius></radius>
</ConstrainedRegions>
<optimizer>GQR</optimizer>
</Goal>
</PostProcessor>
</Models>

<DataObjects>
<PointSet name="myDO">
<Output>X (m), Temperature (K),Y (m)</Output>
</PointSet>
<HistorySet name="TwistDO">
<Input> Power (W), BCTemperature (K)</Input>
<Output>X (m), Temperature (K), Y (m)</Output>
<options>
<pivotParameter>index</pivotParameter>
</options>
</HistorySet>
<DataSet name="outPP">
<Input>InputPlaceHolder</Input>
<Output>Temperature (K), X (m), Y (m)</Output>
<Index var="sensor">X (m), Y (m), Temperature (K)</Index>
</DataSet>
</DataObjects>

<OutStreams>
<Print name="outPP">
<type>csv</type>
<source>outPP</source>
<what>output</what>
</Print>
<Plot name="mySensorPlot">
<plotSettings>
<plot>
<type>scatter</type>
<x>myDO|Output|X (m)</x>
<y>myDO|Output|Y (m)</y>
<cmap>coolwarm</cmap>
<colorMap>myDO|Output|Temperature (K)</colorMap>
</plot>
<plot>
<type>scatter</type>
<x>outPP|Output|X (m)</x>
<y>outPP|Output|Y (m)</y>
<c>black</c>
<marker>x</marker>
</plot>
<xlabel>x</xlabel>
<ylabel>y</ylabel>
</plotSettings>
<actions>
<how>png</how>
</actions>
</Plot>

<Plot name="mySensorPlot3D">
<plotSettings>
<plot>
<type>scatter</type>
<x>TwistDO|Output|X (m)</x>
<y>TwistDO|Output|Y (m)</y>
<z>TwistDO|Output|Temperature (K)</z>
<cmap>coolwarm</cmap>
<colorMap>TwistDO|Output|Temperature (K)</colorMap>
</plot>
<xlabel>x</xlabel>
<ylabel>y</ylabel>
<zlabel>temperature (K)</zlabel>
</plotSettings>
<actions>
<how>png</how>
</actions>
</Plot>
</OutStreams>

</Simulation>
8 changes: 8 additions & 0 deletions tests/framework/PostProcessors/SparseSensing/tests
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,12 @@
required_libraries = 'imageio'
rel_err = 0.01
[../]
[./testSPSLOptiTwistClassification]
type = 'RavenFramework'
input = 'testSPSLOptiTwistClassification.xml'
csv = 'classificationOptiTwist/outPP.csv'
image = 'classificationOptiTwist/mySensorPlot_scatter-scatter.png classificationOptiTwist/mySensorPlot3D_scatter.png'
required_libraries = 'imageio'
rel_err = 0.01
[../]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will only review the image with a relative error of 0.01 which is too small for image comparison. We need to test the csv and the image. rel_err for image can be 0.1

[]