[stub] YAML summary of ReactorNet structure

[ischoegl]

This PR proposes a self-documenting YAML structure for ReactorNet objects that is generated dynamically in the C++ layer via yaml-cpp. While the initial PR simply displays the structure, the idea can be extended to (re-)create ReactorNet objects from YAML files, which would also enable pickling for multiprocessing purposes.

Partially addresses #570

Proposed Changes

• Use YAML to illustrate the structure of a ReactorNet (native using yaml-cpp)
• Incremental changes:
  • Add type and name attributes to ReactorSurface
  • Ensure that automatically generated names are unique (Cython)
  • Ensure that names for FlowDevice, Wall and ReactorSurface are correctly passed between python and C++

Illustration

gas = ct.Solution('h2o2.cti')
reactor = ct.Reactor(gas, name='reactor')
reservoir = ct.Reservoir(gas, name='reservoir')
mfc = ct.MassFlowController(reservoir, reactor, name='mfc')
sim = ct.ReactorNet([reactor])
sim()

produces

reactor-network:
  reactors:
    - reservoir:
        type: Reservoir
        phases: [ohmech]
    - reactor:
        type: Reactor
        phases: [ohmech]
  flow-devices:
    - mfc:
        type: MassFlowController
        reactors: [reservoir, reactor]

More involved setups are avaliable in examples, where ic_engine.py and surf_pfr.py are good test cases as they involve non-trivial ReactorNet configurations.

Thoughts

An extension of this idea that includes simulation parameters and setup details is straight forward, and would allow for serialization/pickling of ReactorNet objects. However, this task remains dependent on some code changes that are beyond the scope of this PR and thus should be handled separately.

Main issues that are beyond the scope of this PR:

• Serialization of Func1 objects
• Differences in Cython Solution handler approach and C++ thermo/kinetics managers.
• The implementation of ReactorSurface is an outlier.

[bryanwweber]

Another possible use of this serialization is to produce a GUI to generate complicated reactor networks.

[codecov]

Codecov Report

Merging #694 into master will decrease coverage by 0.23%.
The diff coverage is 0.71%.

@@            Coverage Diff             @@
##           master     #694      +/-   ##
==========================================
- Coverage   70.63%   70.39%   -0.24%     
==========================================
  Files         372      372              
  Lines       43567    43715     +148     
==========================================
  Hits        30773    30773              
- Misses      12794    12942     +148

Impacted Files	Coverage Δ
include/cantera/zeroD/ReactorNet.h	78.78% <ø> (ø)	⬆️
include/cantera/zeroD/ReactorBase.h	49.01% <ø> (-6.54%)	⬇️
include/cantera/zeroD/ReactorSurface.h	71.42% <ø> (ø)	⬆️
src/zeroD/ReactorSurface.cpp	80.85% <0%> (-3.6%)	⬇️
include/cantera/zeroD/Wall.h	45% <0%> (-5%)	⬇️
src/zeroD/ReactorBase.cpp	56.92% <0%> (-15.63%)	⬇️
src/zeroD/FlowDevice.cpp	69.38% <0%> (-25.06%)	⬇️
src/zeroD/Wall.cpp	61.36% <0%> (-25.74%)	⬇️
src/zeroD/ReactorNet.cpp	55.43% <0%> (-28.16%)	⬇️
include/cantera/zeroD/FlowDevice.h	33.33% <100%> (ø)	⬆️
... and 2 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 265a186...a501589. Read the comment docs.

[codecov]

Codecov Report

Merging #694 (673c43a) into main (6ce695d) will increase coverage by 0.06%.
The diff coverage is 84.31%.

@@            Coverage Diff             @@
##             main     #694      +/-   ##
==========================================
+ Coverage   71.28%   71.35%   +0.06%     
==========================================
  Files         377      377              
  Lines       46335    46488     +153     
==========================================
+ Hits        33032    33171     +139     
- Misses      13303    13317      +14     

Impacted Files	Coverage Δ
include/cantera/zeroD/Reactor.h	72.00% <ø> (ø)
include/cantera/zeroD/ReactorNet.h	81.08% <ø> (ø)
src/zeroD/Reactor.cpp	84.92% <ø> (ø)
src/zeroD/ReactorSurface.cpp	63.93% <16.66%> (-20.52%)	⬇️
include/cantera/zeroD/ReactorSurface.h	69.23% <66.66%> (-2.20%)	⬇️
src/zeroD/ReactorNet.cpp	86.15% <90.90%> (+1.61%)	⬆️
src/zeroD/ReactorBase.cpp	83.58% <95.00%> (+12.15%)	⬆️
include/cantera/zeroD/FlowDevice.h	44.44% <100.00%> (+6.94%)	⬆️
include/cantera/zeroD/ReactorBase.h	70.00% <100.00%> (+4.09%)	⬆️
include/cantera/zeroD/Wall.h	54.54% <100.00%> (+10.10%)	⬆️
... and 7 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6ce695d...bf2eb07. Read the comment docs.

[ischoegl]

Another possible use of this serialization is to produce a GUI to generate complicated reactor networks.

Haven't thought about those possibilities, but I'm sure there are some packages out there that can be leveraged. I was merely interested in testing yaml-cpp to generate yaml descriptions. It's quite straight-forward ...

[speth]

I think that serialization of reactor networks to a YAML format could have a number of useful applications, although I consider pickling for use with the multiprocessing module to be the least of them.

The way I envisioned implementing serialization to YAML, both for this case as well as for Solution objects and 1D flame objects (which also still needs to be implemented) was to work through the AnyMap class as an intermediate. This would allow us to limit most of the YAML-specific code to the methods for converting between YAML and AnyMap, which would make it easier to support additional formats (perhaps HDF5?) or replace the yaml-cpp library if that ever became necessary. It also makes it easier to serialize nested objects without having to convert back and forth between string and YAML::Emitter representations.

[ischoegl]

The way I envisioned implementing serialization to YAML, both for this case as well as for Solution objects and 1D flame objects (which also still needs to be implemented) was to work through the AnyMap class as an intermediate.

OK. It shouldn't be too difficult to convert, now that I have a structure. Any comments on the layout?

PS: on second thought, I do recall that some of the emitters (encode) aren't implemented, so it's not that straight-forward.

[ischoegl]

@speth ... I had another look at AnyMap ... packing information into this structure may be even easier than I thought (the documentation is nice), but the bottle neck appears to be to get it back to YAML.

[ischoegl]

Ad structure:

ReactorNet:
  ReactorBase:
    - reactor:
        type: Reactor
        Phase: 
          - ohmech:
              thermo: IdealGas
              kinetics: Gas
    - reservoir:
        type: Reservoir
        Phase: 
          - ohmech:
              thermo: IdealGas
...

as well as writing ReactorSurface object summaries into each reactor may make more sense. It is not how the code is structured, but it is how things are calculated.

[speth]

@speth ... I had another look at AnyMap ... packing information into this structure may be even easier than I thought (the documentation is nice), but the bottle neck appears to be to get it back to YAML.

Correct, the AnyMap to YAML converter has not yet been written. My plan had been to write that in conjunction with the Solution to YAML serializer, since it's easiest to write when you have a use case immediately at hand.

[speth]

In terms of the structure, you could flatten it a bit. Given that the names for each component have to be unique in order to make the connections, you can just use a mapping instead of a list of single-key mappings, e.g.

ReactorNet:
  ReactorBase:
    reactor:
      type: Reactor
      phase: ohmech
      thermo.type: IdealGas
      kinetics.type: Gas
    reservoir:
      type: Reservoir
      phase: ohmech
      thermo.type: IdealGas

You could also flatten it another level if you don't worry about separating objects of different types into different groups, but I can see how that does simplify the implementation of calling the factory functions, so perhaps retaining that is worthwhile. I would suggest that names here follow patterns more like what is done in the YAML format, and not duplicate jargon from the implementation, e.g. instead of ReactorBase and FlowDevice, I would got with reactors and flow-devices as the names for these sections.

I'm also wondering how to encode all of the information that is needed to reconstruct the network.

For the Solution attached to a Reactor, the most compact way to serialize that would be just the input file name, phase name, and state, if the phase was constructed from a file and the species/reactions have not been modified. But that information isn't currently available from a Solution object. Once the full serialization of Solution objects is implemented, you could just use that, but it would be kind of inefficient for the case where you just want to use an existing input file for the phase definition.

Serialization of user-defined functions will also be a pretty substantial challenge, without reverting to something like the more difficult to use "building block" style ones that were used in the old Python module, and are still used in the Matlab toolbox.

[ischoegl]

@speth ... thanks for the feedback. I'll think about it some more. Regarding Solution, I just posted a question in #695 that summarized my thoughts on that.

My plan had been to write that in conjunction with the Solution to YAML serializer, since it's easiest to write when you have a use case immediately at hand.

I wrote this PR mostly as I wanted to have a simple test case to understand serialization. I can create AnyMap representations with fairly little effort which would be another use case ...

[ischoegl]

@speth ... 👍 on skipping sequences and using plain names. Anticipating #696 (which will guarantee unique names for C++ SolutionBase objects), the resulting structure would be:

reactor-network:
  reactors:
    reactor:
      type: Reactor
      phase: [ohmech_1]
    reservoir:
      type: Reservoir
      phase: [ohmech_1]
  flow-devices:
    mfc:
      type: MassFlowController
      in: reservoir
      out: reactor

Also, I agree that serialization of user defined functions may be non-trivial. Haven't looked into this (and probably shouldn't for the time being.) At least for the moment, it appears to be low priority.

PS: the lists for phase entries are motivated by ReactorSurface ... which imho should be lumped into the reactor entry. The implementation is an outlier ...

[ischoegl]

Pushed a revision with some changes. I opted to retain the list entries for named objects to differentiate between groups and what are entries (it should be easily be parsed by human eyes). The output for the surf_pfr.py example would look as follows:

reactor-network:
  reactors:
    - downstream:
        type: Reservoir
        phases: [gas]
    - upstream:
        type: Reservoir
        phases: [gas]
    - IdealGasReactor_1:
        type: IdealGasReactor
        phases: [gas, Pt_surf]
  flow-devices:
    - PressureController_1:
        type: PressureController
        reactors: [IdealGasReactor_1, downstream]
    - MassFlowController_1:
        type: MassFlowController
        reactors: [upstream, IdealGasReactor_1]

[ischoegl]

Renaming to ‘stub’ as there is a clear dependence on YAML work and questions about surface kinetics (#699) remain.

[joebarteam11]

Hello
Is this PR still under development? The possibility to use multiprocessing with reactor networks and free flames is very interesting.
Maybe I can help, if you can give some guidelines ;)
Thanks!

[ischoegl]

@joebarteam11 ... thanks for your question; the answer for whether this is still pursued really is 'it depends'.

I am currently working on an improved way of serializing simulation output (see #1426, although it's not necessarily evident). For oneD objects like FreeFlame, it is already possible to save all settings to HDF or YAML files and recreate the exact same simulation settings (note that 3.0 will bring some updates to this, see #1385); once the current PR is done, my next step will be to do the same for zeroD like ReactorNets. The YAML summary draft in this PR will be a template and likely survive in some form ...

[joebarteam11]

Thanks for your reply @ischoegl, I can't wait to check out these new features!
All the best