Modify EnergyPlus structure to expose it as a callable shared library

[Myoldmopar]

This is starting to look very promising.

• Code compiles on Linux into libEnergyPlus.so
• Added a dummy program.cc source that links to this dynamically and runs EnergyPlus fine when linking to void EnergyPlusPgm()
• Added callback functions to the dummy program, they properly register with E+ and after some fixes, E+ properly calls back.

Thus everything works perfectly on Linux.

ToDo:

• Make it cross platform
• Verify CMake definition logic
• Clean up the #defines in the code, verify with specific team members
• Clean up test executable

[Myoldmopar]

@mjwitte @kbenne @lefticus @DeadParrot @nealkruis @lgentile

Summary

This pull request would bring in the code necessary to build EnergyPlus as a shared library. This is how GUIs like DesignBuilder prefer to run EnergyPlus, as it lets them register callbacks within EnergyPlus, which EnergyPlus then calls back to with updates on messages and simulation progress.

Changes

The changes made include:

• Adding a MAKE_ENERGYPLUS_LIBRARY CMake option here
• Adding some CMake logic that will build E+ differently if it is building it as a library here
  • Instead of ADD_EXECUTABLE(EnergyPlus ..., it will ADD_LIBRARY(EnergyPlus SHARED ...
  • On Unix it adds the -fPIC flag so that it produces position independent code
  • It also compiles a small test program that links to this shared library to test out that it works
• Adds the necessary shared library specific variables to DataGlobals here
  • This includes the void pointers to the callback functions
• Updated DisplayRoutines to callback to these functions when enabled here
• Enabled the calls to DisplaySimDaysProgress in the SimulationManager and the WeatherManager which had been commented because they didn't do anything before this
• Made quite a few changes in EnergyPlus/main.cc
  • Added platform-specific #includecommands to get access to the chdir and _chdir commands
  • Added platform-specific code to #define the EXPORT symbol to _declspec on Windows, or to blank on other platforms.
  • Added conditional to either define int main() as before or void EnergyPlusPgm if building as a library
  • Added logic to change working directory to the one specific by the argument to the library function EnergyPlusPgm
  • Add Callback storage functions to hold pointers to the functions to be called to provide feedback during the simulation
• Added a test program that will link against this library. This program expects a single command line argument, and that is a directory path that contains Energy+.idd, in.idf, and if needed in.epw. This is how the library has worked for interfaces in the past.

Status

I have tested this on Windows and Linux. Previously this had never been available for anything but Windows, so this is actually an enhancement. Right now the library code should be isolated from any non-library operation, so there should be no issue there. It works really great with the test program I put in there, so at this point, I would just like some of you to provide feedback on the #includes, the CMake rules, etc., in case there is anything I missed. I'm not looking to really add any more functionality, as this satisfies the capabilities from before the C++ translation, and as yet I haven't had to invest any significant amount of time, and I'd like to keep it that way.

[DeadParrot]

@Myoldmopar this seems good to me -- no obvious red flags. For this kind of EnergyPlus-as-a-service use the typical problems of throwing exceptions and heap management across DLL boundaries should not be an issue.

[Myoldmopar]

Thanks a lot @DeadParrot! In that case, @mjwitte, I think this is ready to
go.

[mjwitte]

Um, OK, I'm not sure I know how to review this, so handing it off to @kbenne or @nealkruis

[Myoldmopar]

Well, I don't know what that whole unhandled elemental failure is about (@lefticus), but in any case, I would say the review process could be:

• Code review (@DeadParrot offered some comments)
• Build the library: Make a new build folder, run cmake, enable MAKE_ENERGYPLUS_LIBRARY, and build the EnergyPlus project; Make sure it builds the DLL
• Build the wrapper program: build the Test_EnergyPlus_As_Library project, it should create an executable in build/Products.
• Test the wrapper program: stick an in.idf, Energy+.idd, and perhaps in.epw in a folder somewhere. Open your command prompt and run the test program: $ Test_EnergyPlus_As_Library Path\To\Run\Folder

[Myoldmopar]

@kbenne Do you have time to verify this? If not I'll find someone else to give it a try. Thanks!

[nrel-bot]

@kbenne @Myoldmopar @lgentile it has been 7 days since this pull request was last updated.

[Myoldmopar]

Just merged develop into this branch to get it to pick up the parametric processor fixes, but this should be good to go. @kbenne ?

[Myoldmopar]

@kbenne CI is pleased with this branch now; care to merge? Any reason to wait?

[Myoldmopar]

@mjwitte @rraustad @kbenne @lefticus @DeadParrot @lgentile

Overview

I figure this could involve you guys quite a bit, so I wanted to give you a heads up as to the status here. While in the past we had built the EnergyPlus DLL as a little side project, we decided moving forward, it would be better to include it directly in the build/test process so it is always available and always tested.

History

We have been, for a few months now, building EnergyPlus as a static library first, then building the executable directly against this. It worked well, because unit tests could be linked to the static library. Since we were building a shared library (dll) now, we decided, OK we'll just convert that static library into a dynamic library. However, this was problematic. When the library is dynamic, all the symbols (variables, functions) that are needed by unit tests must be exported. @lefticus did this here. With our few (6) unit tests, this started revealing an ugly pattern. Pretty soon variables and functions across the code would require this decoration to be unit tested.

New Process

So, we decided on a little longer list of build steps. I must note here that with CMake creating your projects, you will be almost not affected, which is awesome. Here are the build steps now:

• First a static library is built, linking together all of the core EnergyPlus source code. Unit tests are linked against this. This library is called EnergyPlusCore. In Visual Studio, this is the project you will expand to see the EnergyPlus source code.
• Second the shared library is built, linking together the static library and EnergyPlusPgm.cc, which exposes the public API. You'll never really debug this or mess with this. Unless you add something to the public API. This is now called EnergyPlusLib. This shared library (dll) is packaged with EnergyPlus now.
• Third the EnergyPlus binary is built, which is a super thin wrapper over the shared library. This is the project you will actually run/debug. It will call out to the dll, though you won't really know it. This is simply called EnergyPlus.

Action items

• Nothing for now.
• Once this is merged into develop, and you update your clone, you'll need to likely adjust one thing. In Visual Studio, you'll want to rebuild your project, which will cause CMake to run, then Visual Studio will ask if you want to reload your projects. Yes. Then just make sure the EnergyPlus project is set as the default startup project.
• That's it, no other changes necessary. If you browse to the build folder, you should see a .lib static library, a .dll shared library, and the actual EnergyPlus.exe binary. If you want to run EnergyPlus in a different folder for some reason, just make sure you grab all the .dll's to take along.

Status

• This needs a little review and testing. It works well on my Linux box with the last commit, but I haven't tried Windows or Mac yet. I am pretty hopeful there won't be any issue. I'd like feedback from interested parties in naming of each project. In a sense I'd like to name them all just EnergyPlus, since they are different target types, but I didn't check whether CMake can handle that or not.
• This needs a unit test added on. Currently a simple test program is built that links against the dll and tests out the callback functions, but this isn't included in the testing framework. Once that is in place it should be ready to get merged in. I'm working on that now while CI tests across platforms.

[kbenne]

Minor quibble. I suggest EnergyPlusLib for the static library and EnergyPlusAPI for the dll.

[Myoldmopar]

@kbenne Quibble understood. I'll update those names and also try to implement a CI test for this work. Then I'll merge develop in to resolve whatever merge conflict occurred and this should be ready to merge later today.

[kbenne]

@Myoldmopar I just happened to be working on this branch myself. I have resolved the conflict and will push momentarily.

[Myoldmopar]

Bonus!

[mjwitte]

@kbenne So, if I'm following correctly, the EnergyPlus install folder will contain a tiny EnergyPlus.exe and a large EnergyPlusAPI.dll with the API and all the rest of EnergyPlus? EnergyPlusLib seems to describe the contents better that EnergyPlusAPI. Or am I misunderstanding your quibble.

[Myoldmopar]

@kbenne Any idea why this diff is showing up? I don't recall adding this, but it thinks I did anyway.

[kbenne]

@Myoldmopar not sure.

[kbenne]

@mjwitte you are understanding correctly. The build directory on the other hand will have a large static library, an equivalently sized DLL due to that fact that the static library is linked in, and a small execrable. This PR does not explicitly install the static library, it has little value besides providing a compilation unit to link the tests to.

[kbenne]

@mjwitte Oh and the root of my minor quibble is that I believe core is rather vague and arbitrary. It becomes ambiguous as to what constitutes "core". We used the term in OpenStudio and I think looking back now it seem rather silly. I was just suggesting, recognize the arbitrariness and use words that indicate something more concrete. Such as these things are public (meaning we support their consumption by our clients - API) and these other things are private, meaning you should not even need to know they exist as a client. The physical size doesn't really factor in. Yes the API is small in number of functions, but it happens that those functions (run program) pack a wallop.

[mjwitte]

@kbenne Whatever you and @Myoldmopar have agreed upon is fine with me.

[Myoldmopar]

@kbenne Will you be able to merge this in or should I gather a different dev?

[eayoungs]

I'm very curious about how this feature works. Is there any further documentation or examples of using the API?

Thanks

[Myoldmopar]

Not as of right now, outside of the discussion above. The development team has goals to expose more underlying energy plus stuff in the api, but right now that's just in the plans. If you want to use the energypluspgm public function mentioned above just write back here and I can add detail.

[eayoungs]

Thanks Myoldmopar; will do.