Add Utility for Generating Alfalfa Metadata to OpenStudio #5236

kbenne · 2024-08-16T15:41:55Z

Alfalfa is a platform which allows users to interact with OpenStudio/EnergyPlus models in real time. In order to accomplish this users must specify what components of the model to connect with Alfalfa as input and output points. This PR moves the mechanisms for creating "Alfalfa Points" into the OpenStudio runner so that they may be accessed during the measure workflow.

Pull Request Author

Model API Changes / Additions
Any new or modified fields have been implemented in the EnergyPlus ForwardTranslator (and ReverseTranslator as appropriate)
Model API methods are tested (in src/model/test)
EnergyPlus ForwardTranslator Tests (in src/energyplus/Test)
If a new object or method, added a test in NREL/OpenStudio-resources: Add Link
If needed, added VersionTranslation rules for the objects (src/osversion/VersionTranslator.cpp)
Verified that C# bindings built fine on Windows, partial classes used as needed, etc.
All new and existing tests passes
If methods have been deprecated, update rest of code to use the new methods

Labels:

If change to an IDD file, add the label IDDChange
If breaking existing API, add the label APIChange
If deemed ready, add label Pull Request - Ready for CI so that CI builds your PR

Review Checklist

This will not be exhaustively relevant to every PR.

Perform a Code Review on GitHub
Code Style, strip trailing whitespace, etc.
All related changes have been implemented: model changes, model tests, FT changes, FT tests, VersionTranslation, OS App
Labeling is ok
If defect, verify by running develop branch and reproducing defect, then running PR and reproducing fix
If feature, test running new feature, try creative ways to break it
CI status: all green or justified

TShapinsky · 2024-08-20T17:04:31Z

@kbenne what do you think about the language of add vs something like export.

Right now all the AlfalfaJSON methods are phrased as add like addGlobalVariable, addMeter, etc. I wonder if it might make more sense to replace add with export since what you're doing is exporting a GlobalVariable, Meter, OutputVariable, etc. to Alfalfa so it may be consumed there.

TShapinsky · 2024-09-04T21:20:23Z

@kbenne this is pretty much there.
My TODO:

Finish json serialization saving and connection with existing OS workflow.
Audit if there are additional Model Objects of IDF objects that we should support automatic point creation for
Add methods for exposing lists of variables from one object, as seen in EnergyManagementSystem:GlobalVariable

kbenne · 2024-09-06T17:14:06Z

@TShapinsky as we discussed, it would be a good idea to test a workflow that excerses this new feature. You can see an example of workflow tests here. Note they are not Google Tests, but instead these are implemented purely in the CTest wrapper.

kbenne · 2024-09-06T17:18:08Z

I created a new issue to create documentation for this new API.

TShapinsky · 2024-09-13T20:08:40Z

@kbenne I'm happy where everything is, with the exception of the behaviour for dealing with duplicate ids. One possible approach is to modify duplicate ids on export so they become unique. This would work, but it could be confusing and it is also non-deterministic. If someone was expecting a certain set of IDs from a measure they might not get them if a previous measure adds points of the same id.

Another approach is to throw an error on export which could be annoying. It also runs the possibility of weird behaviour where based on naming of model objects certain models would error while others didn't. Are names unique across an IDF? or are they just unique to an idd type?

A possible third approach would be to refactor the id out of the point and have it be set in the AlfalfaJSON. This is probably the most correct as it is a key used to refer to the point and thus not something that the point needs to know about. This would allow for runtime checking of point Id against the existing points.

I think the best option may be a combination of the first and third. Refactoring the point id to be stored in the AlfalfaJSON object and dynamically renaming point ids to prevent overlap. A warning could also be printed to alert of this happening. But that would only be shown locally, so it wouldn't include weird behaviour that happens when the measures interact with the baked in alfalfa measures.

Maybe after this we need to reevaluate if we want to be automatically baking points into alfalfa or if we should just provide the measures we create for generating generic points as a resource for people putting together workflows. That would allow for the maximum amount of reproducibility between alfalfa and local development.

kbenne · 2024-09-19T14:22:53Z

@jmarrec I would grateful to have you weigh in on this. The challenge is that I'm not sure that you have enough context or any available time. If you don't have time, that's understandable, just say so. My intention is to merge this in time for the next release unless there are any serious red flags that emerge.

cc @DavidGoldwasser @TShapinsky

kbenne · 2024-09-19T14:23:33Z

@wenyikuang can you investigate the CI failures?

wenyikuang · 2024-09-19T16:39:27Z

Looks the windows-incremental is lost connection from the jenkins, let me look

TShapinsky · 2024-09-19T17:06:40Z

If i remember correctly, the osx build fails were from tests failing for BCL measures.

kbenne · 2024-09-19T18:10:10Z

python/engine/test/AlfalfaMeasure/measure.py

+        alfalfa.exposeConstant(10, "safe value")
+        alfalfa.exposeMeter("Facility:Electricity", "Facility Electricity")
+        alfalfa.exposeActuator("somehting", "another thing", "key", "example actuator")
+        alfalfa.exposeOutputVariable("Whole Building", "Facility Total Purchased Electricity Energy", "output variable")


Is this a change? I seem to recall that these originally took ModelObject instances. I think when we @TShapinsky last spoke you were contemplating a change.

A risk I can think of is that the final names might not be known at ModelMeasure time.

resources/Examples/compact_osw/measures/AlfalfaEPlusPython/measure.py

resources/Examples/compact_osw/measures/AlfalfaModelPython/measure.py

src/utilities/CMakeLists.txt

wenyikuang · 2024-09-19T21:18:23Z

Resolve the issue of connection between jenkins and windows-incre-on-aws

https://ci.openstudio.net/blue/organizations/jenkins/openstudio-incremental-windows/detail/PR-5255/7/pipeline

(From another PR but using the same node)

Or are you mentioning these one?

The following tests FAILED:

	665 - BCLFixture.RemoteBCL_BCLSearchResult (Failed)

	1877 - ModelFixture.PythonPluginInstance_NotPYFile (Failed)

	2304 - ModelFixture.ScheduleFile (Failed)

	3771 - OpenStudioCLI.Check_Alfalfa (Failed)

jmarrec

This is good work. I'm just generally picky.

There are a couple of constructs I'd change, I'd also suggest taking your files for a spin with clang-tidy to modernize the code a bit and follow best practices.

My main point is more around the articulation of the classes between themselves, and what public API should be like (probably use actual type values to store stuff and not JSON, provide getters, some methods should probably be private, and error handling should be reworked a bit).

python/engine/test/AlfalfaMeasure/measure.py

python/engine/test/PythonEngine_GTest.cpp

python/module/openstudio.py

resources/Examples/compact_osw/measures/AlfalfaEPlusPython/measure.py

resources/Examples/compact_osw/measures/AlfalfaModelPython/measure.py

src/utilities/filetypes/AlfalfaJSON.cpp

jmarrec · 2024-09-20T12:49:11Z

src/utilities/filetypes/AlfalfaJSON.cpp

+    m_impl->exposePoint(point);
+  }
+
+  boost::optional<AlfalfaPoint> AlfalfaJSON::exposeFromObject(const openstudio::IdfObject& idf_object, const std::string& display_name) {


THe vast majority of the code I've read was "throwy", but here you go with LOG and return boost::none. That's fine, just noting.

The general pattern was I log and return boost::none everywhere possible, but in ctors you can't do that (to my knowledge) to I throw a std::runtime_error.

src/utilities/filetypes/AlfalfaJSON.cpp

src/utilities/filetypes/AlfalfaJSON.hpp

src/workflow/RunPostProcess.cpp

python/engine/test/PythonEngine_GTest.cpp

jmarrec · 2024-09-20T13:07:34Z

@wenyikuang cppcheck is broken, you should fix that: https://github.com/NREL/OpenStudio/actions/runs/10853524579/job/30432985755#step:3:16

src/utilities/filetypes/AlfalfaJSON.hpp

src/utilities/filetypes/AlfalfaJSON.cpp

src/utilities/alfalfa/AlfalfaPoint_Impl.hpp

jmarrec · 2024-09-20T14:21:39Z

Made a few cosmetic changes in [Developer] Adjust alfalfa_utility #5259

…alfa measure

TShapinsky · 2024-10-11T21:16:38Z

@jmarrec I'm starting to get into the cleaning phase. I think it would be useful if you could go over the Component code (ComponentBase, AlfalfaComponent, AflalfaActuator, AlfalfaConstant, AlfalfaMeter, AlfalfaOutputVariable, AlfalfaGlobalVariable) and let me know what you think. I'm decently happy with the new architecture, but it isn't exactly something that is elsewhere in the code base so I'd like your opinion.

ci-commercialbuildings · 2024-10-17T18:45:47Z

CI Results for f54b39d:

Ubuntu 20.04:
Ubuntu 22.04:
- OpenStudio-3.8.0+f54b39d292-Ubuntu-22.04-x86_64.deb
- OpenStudio-3.8.0+f54b39d292-Ubuntu-22.04-x86_64.tar.gz
Darwin:
- OpenStudio-3.8.0+f54b39d292-Darwin-x86_64.dmg
- OpenStudio-3.8.0+f54b39d292-Darwin-x86_64.tar.gz
Windows:
Code Coverage:

jmarrec · 2024-10-29T11:19:53Z

python/module/openstudio.py

@@ -28,6 +28,7 @@
    from . import openstudioosversion as osversion
    from . import openstudioradiance as radiance
    from . import openstudiosdd as sdd
+    from . import openstudioualfalfa as alfalfa


what's that extra "u" in there? I think you meant openstudioalfalfa @TShapinsky

jmarrec · 2024-10-29T11:22:08Z

src/alfalfa/AlfalfaComponent.hpp

+    template <typename T, std::enable_if_t<std::is_base_of<ComponentBase, T>::value, bool> = true>
+    AlfalfaComponent(T component) : m_component(std::make_unique<T>(std::move(component))) {}


seeing this while looking diagonally at this PR changes since I was out of office. Makes me wonder if I should investigate what's up or not. @kbenne I'm assuming you reviewed the changes already while I wasn't around and I don't need to?

I'm failing to see directly why this AlfalfaComponent class is needed. Instead of just keeping ComponentBase (renamed to AlfalfaComponent probably, but basically an abstract class with the pure virtual methods and maybe a couple of other defaulted virtual methods that would define that common API) and just be done with it... That "interface" templated class seems superfluous (and overcomplicated) at first glance

I think I was getting compiler errors with the full virtual abstract base class. But I'll do a quick test to see exactly what the issue was. Maybe there's a better solution.

Ahh, the biggest issue appears to be that you can't boost::optional an abstract base class. I can obviously use a unique_ptr but I don't think that translates to swig very well...

@jmarrec It wouldn't hurt to have your touch on this for one more round. If you and @TShapinsky agree to a simplification then I'm all for that.

Right. We don't have that problem with abstract base classes in model namespace for eg because of the pImpl pattern...
I'll take a quick look tomorrow to check if I see a simpler way of doing it, otherwise this is fine.

(We should fix the python import I reported though)

jmarrec · 2024-10-29T14:34:26Z

src/alfalfa/AlfalfaJSON_Impl.hpp

+
+      void exposePoint(const AlfalfaPoint& point);
+
+      std::vector<AlfalfaPoint> points();


should be const. cppcheck catches this (now that I fixed the workflow on a separate PR...)

kbenne self-assigned this Aug 16, 2024

TShapinsky changed the title ~~Basic structure for Alfalfa OpenStudio integration~~ Add Utility for Generating Alfalfa Metadata to OpenStudio Aug 16, 2024

kbenne mentioned this pull request Sep 6, 2024

Add documentation for new Alfalfa related APIs #5250

Open

TShapinsky added Pull Request - Ready for CI This pull request if finalized and is ready for continuous integration verification prior to merge. component - Ruby bindings component - Python bindings labels Sep 6, 2024

kbenne commented Sep 19, 2024

View reviewed changes

resources/Examples/compact_osw/measures/AlfalfaEPlusPython/measure.py Show resolved Hide resolved

kbenne commented Sep 19, 2024

View reviewed changes

resources/Examples/compact_osw/measures/AlfalfaModelPython/measure.py Show resolved Hide resolved

kbenne commented Sep 19, 2024

View reviewed changes

src/utilities/CMakeLists.txt Outdated Show resolved Hide resolved

jmarrec requested changes Sep 20, 2024

View reviewed changes

jmarrec reviewed Sep 20, 2024

View reviewed changes

python/engine/test/PythonEngine_GTest.cpp Outdated Show resolved Hide resolved

jmarrec reviewed Sep 20, 2024

View reviewed changes

src/utilities/filetypes/AlfalfaJSON.hpp Outdated Show resolved Hide resolved

jmarrec reviewed Sep 20, 2024

View reviewed changes

src/utilities/filetypes/AlfalfaJSON.cpp Outdated Show resolved Hide resolved

src/utilities/filetypes/AlfalfaJSON.cpp Outdated Show resolved Hide resolved

src/utilities/alfalfa/AlfalfaPoint_Impl.hpp Outdated Show resolved Hide resolved

jmarrec mentioned this pull request Sep 20, 2024

[Developer] Adjust alfalfa_utility #5259

Merged

19 tasks

TShapinsky added 3 commits October 10, 2024 16:07

Basic structure for Alfalfa OpenStudio integration

9843b0f

do formatting

3cd4267

Add ability to create alfalfa points directly from openstudio objects'

41dbcc9

jmarrec and others added 11 commits October 10, 2024 16:07

Raw string litteral and absolute minimal testing for PythonEngine alf…

41abf4a

…alfa measure

Fixup AlfalfaJSON code

9ed8e01

Lint AlfalfaPoint

578381e

Lint up the rest

29f5b9a

move alfalfa to top level and reimagine Component implementation

a3f527d

getPoints() to points()

28a9c80

formatting

c5c0cb1

reset utilities cmakelists now that we are not using it

f2c2956

commit alfalfajson_gtest

1677007

move checks for object type into components

ff2ff6f

formatting

66b50e3

TShapinsky force-pushed the alfalfa_utility branch from 810224f to 66b50e3 Compare October 10, 2024 22:07

TShapinsky added 6 commits October 11, 2024 09:32

fix dll export

3014739

undo overzealous clang-format

be526c3

undo overzealous clang-format

a221335

undo overzealous clang-format

3cd641b

use simplified API header

662ddbb

omit clone() from swig

1de9181

TShapinsky added 2 commits October 15, 2024 12:45

cleanup imports and clang-tidy

5fe1207

try using runtime wrapping for make fmt happy on ubuntu 22.04

d1f6cff

DavidGoldwasser added this to the OpenStudio SDK 3.9.0 milestone Oct 17, 2024

TShapinsky added 2 commits October 17, 2024 12:00

fix alfalfa ctests

5015251

fix linking of alfalfa ruby bindings

f54b39d

kbenne merged commit 8303235 into develop Oct 21, 2024
1 of 6 checks passed

kbenne deleted the alfalfa_utility branch October 21, 2024 14:28

jmarrec reviewed Oct 29, 2024

View reviewed changes

anyaelena mentioned this pull request Nov 5, 2024

Move metadata generation (Alfalfa Ruby gem) into core OS NREL/alfalfa#506

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Utility for Generating Alfalfa Metadata to OpenStudio #5236

Add Utility for Generating Alfalfa Metadata to OpenStudio #5236

kbenne commented Aug 16, 2024 •

edited by TShapinsky

Loading

TShapinsky commented Aug 20, 2024

TShapinsky commented Sep 4, 2024 •

edited

Loading

kbenne commented Sep 6, 2024

kbenne commented Sep 6, 2024

TShapinsky commented Sep 13, 2024 •

edited

Loading

kbenne commented Sep 19, 2024

kbenne commented Sep 19, 2024

wenyikuang commented Sep 19, 2024

TShapinsky commented Sep 19, 2024

kbenne Sep 19, 2024

kbenne Sep 19, 2024

wenyikuang commented Sep 19, 2024

jmarrec left a comment

jmarrec Sep 20, 2024

TShapinsky Sep 27, 2024

jmarrec commented Sep 20, 2024

jmarrec commented Sep 20, 2024

TShapinsky commented Oct 11, 2024

ci-commercialbuildings commented Oct 17, 2024 •

edited

Loading

jmarrec Oct 29, 2024

jmarrec Oct 29, 2024

jmarrec Oct 29, 2024 •

edited

Loading

TShapinsky Oct 30, 2024

TShapinsky Oct 30, 2024

kbenne Oct 30, 2024

jmarrec Oct 30, 2024

jmarrec Oct 29, 2024

		template <typename T, std::enable_if_t<std::is_base_of<ComponentBase, T>::value, bool> = true>
		AlfalfaComponent(T component) : m_component(std::make_unique<T>(std::move(component))) {}


		void exposePoint(const AlfalfaPoint& point);

		std::vector<AlfalfaPoint> points();

Add Utility for Generating Alfalfa Metadata to OpenStudio #5236

Add Utility for Generating Alfalfa Metadata to OpenStudio #5236

Conversation

kbenne commented Aug 16, 2024 • edited by TShapinsky Loading

Pull Request Author

Review Checklist

TShapinsky commented Aug 20, 2024

TShapinsky commented Sep 4, 2024 • edited Loading

kbenne commented Sep 6, 2024

kbenne commented Sep 6, 2024

TShapinsky commented Sep 13, 2024 • edited Loading

kbenne commented Sep 19, 2024

kbenne commented Sep 19, 2024

wenyikuang commented Sep 19, 2024

TShapinsky commented Sep 19, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wenyikuang commented Sep 19, 2024

jmarrec left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jmarrec commented Sep 20, 2024

jmarrec commented Sep 20, 2024

TShapinsky commented Oct 11, 2024

ci-commercialbuildings commented Oct 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jmarrec Oct 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kbenne commented Aug 16, 2024 •

edited by TShapinsky

Loading

TShapinsky commented Sep 4, 2024 •

edited

Loading

TShapinsky commented Sep 13, 2024 •

edited

Loading

ci-commercialbuildings commented Oct 17, 2024 •

edited

Loading

jmarrec Oct 29, 2024 •

edited

Loading