Usage of generated FMU with external tool such as Dymola or similar

[alfoa]

Quick question. Did you guys have some advices on how to use a PythonFMU generate FMU in platforms like Dymola? Or on how to debug the FMU in such (or for such) platforms?
I was able to create an FMU and use it in python based importers (e.g. FMPy) but the same FMU can not be used in Dymola. The error I get is very "undescriptive":

Translation of ROMSlave_fmu:

The DAE has 6 scalar unknowns and 6 scalar equations.

The model has direct feed through from the inputs to the outputs.

Statistics

Selected continuous time states

Some libraries are only present as 64-bit:
   C:/Users/ALFOA/Documents/Dymola/Resources/Library/FMU/ROMSlave/binaries/win64
automatically switching to that mode.
If this is not desired use Advanced.CompileWith64 to control linking.

Finished

Process Simulating: Initialization
 stopped without any error indication.

Simulating: Initialization

Not possible to open file "ROMSlave_fmu.mat": No such file or directory

ERRORS have been issued.

Any advice?

[markaren]

I don't really have any good suggestions. There are so many things that can go wrong with these complex applications. I personally only use PythonFMU through "simple" importers. Dymola might use it's own Python version (?), which might cause problems..

I guess the best way to debug is to build the shared library (and modify it accordingly) for your platform. This is straightforward. Just modify the C++ code and run cmake. There are some helper scripts for that. Then run pip install . to get a pythonfmu version bundling the updated shared library. It could help you figure out when the error happens.

[alfoa]

I used the FMUComplianceChecker and we get the following error:

Fatal Python error: initfsencoding: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings

put the in PATH and PYTHONPATH we link to the right pythons etc. (and in the interpreter no problem).

[markaren]

I have no issues with the compliance checker on my PC. On Path I have
C:\Users...\AppData\Local\Programs\Python\Python37
C:\Users...\AppData\Local\Programs\Python\Python37\Scripts\

I honestly don't know if PYTHONPATH has an effect when Python is loaded as a C library.

[alfoa]

Hi. Actually it works. I am so sorry...I forgot to add the PYTHONHOME environment variable. I am sorry for this.

[alfoa]

@markaren I am sorry to bother you again. The simple FMU works fine wiht Dymola and/or the Compliance Checker.
Unfortunately when I try to import numpy (at the top of the module) I get a segmentation fault

[VERBOSE][FMUCHK] Setting log level to [DEBUG]
[VERBOSE][FMUCHK] This binary is build without debug log messages.
 Reconfigure with FMUCHK_ENABLE_LOG_LEVEL_DEBUG=ON and rebuild to enable debug level logging
[INFO][FMUCHK] FMI compliance checker Test [FMILibrary: Test] build date: Mar  5 2021
[INFO][FMUCHK] Called with following options:
[INFO][FMUCHK] C:\Users\ALFOA\projects\FMUComplianceChecker\build\Debug\fmuCheck.win64.exe -l 6 ../../../raven/test_turbine_fmi/PythonFMU/ROMSlave.fmu
[INFO][FMUCHK] Will process FMU ../../../raven/test_turbine_fmi/PythonFMU/ROMSlave.fmu
[VERBOSE][FMILIB] Allocating FMIL context
[VERBOSE][FMILIB] Detecting FMI standard version
[VERBOSE][FMIZIP] Unpacking FMU into C:\Users\ALFOA\AppData\Local\Temp\1\fmucktmpa21480
[VERBOSE][FMIXML] Parsing XML to detect FMI standard version
[VERBOSE][FMIXML] XML specifies FMI 2.0
[INFO][FMILIB] XML specifies FMI standard version 2.0
[VERBOSE][FMILIB] Parsing model description XML
[VERBOSE][FMI2XML] Parsing XML element fmiModelDescription
[VERBOSE][FMI2XML] Parsing XML element CoSimulation
[VERBOSE][FMI2XML] Parsing XML element LogCategories
[VERBOSE][FMI2XML] Parsing XML element ModelVariables
[VERBOSE][FMI2XML] Building alias index
[VERBOSE][FMI2XML] Parsing XML element ModelStructure
[VERBOSE][FMI2XML] Parsing XML element Outputs
[VERBOSE][FMILIB] Parsing finished successfully
[INFO][FMUCHK] Model name: ROMSlave
[INFO][FMUCHK] Model GUID: 7ee018ba-7e0f-11eb-9804-54bf645b8fcc
[INFO][FMUCHK] Model version:
[INFO][FMUCHK] FMU kind: CoSimulation
[INFO][FMUCHK] The FMU contains:
0 constants
2 parameters
0 discrete variables
2 continuous variables
1 inputs
1 outputs
0 local variables
0 independent variables
0 calculated parameters
2 real variables
0 integer variables
0 enumeration variables
0 boolean variables
2 string variables

[INFO][FMUCHK] No input data provided. In case of simulation initial values from FMU will be used.
[INFO][FMUCHK] Printing output file header
[INFO][FMUCHK] Model identifier for CoSimulation: ROMSlave
[INFO][FMILIB] Loading 'win64' binary with 'default' platform types
[VERBOSE][FMICAPI] Loaded FMU binary from C:\Users\ALFOA\AppData\Local\Temp\1\fmucktmpa21480\binaries\win64\ROMSlave.dll
[VERBOSE][FMICAPI] Loading functions for the co-simulation interface
[VERBOSE][FMILIB] Successfully loaded all the interface functions
[INFO][FMUCHK] Version returned from CS FMU:   2.0
[VERBOSE][FMICAPI] Calling fmi2GetModelTypesPlatform
[VERBOSE][FMUCHK] allocateMemory( 1, 192) called. Returning pointer: 0000020298623EE0
[VERBOSE][FMUCHK] allocateMemory( 1, 160) called. Returning pointer: 000002029A8B3C70
Segmentation fault

If I move the import inside the do_step method, I get the following error (captured from the Compliance Checker):

[VERBOSE][FMUCHK] Setting log level to [DEBUG]
[VERBOSE][FMUCHK] This binary is build without debug log messages.
 Reconfigure with FMUCHK_ENABLE_LOG_LEVEL_DEBUG=ON and rebuild to enable debug level logging
[INFO][FMUCHK] FMI compliance checker Test [FMILibrary: Test] build date: Mar  5 2021
[INFO][FMUCHK] Called with following options:
[INFO][FMUCHK] C:\Users\ALFOA\projects\FMUComplianceChecker\build\Debug\fmuCheck.win64.exe -l 6 ../../../raven/test_turbine_fmi/PythonFMU/ROMSlave.fmu
[INFO][FMUCHK] Will process FMU ../../../raven/test_turbine_fmi/PythonFMU/ROMSlave.fmu
[VERBOSE][FMILIB] Allocating FMIL context
[VERBOSE][FMILIB] Detecting FMI standard version
[VERBOSE][FMIZIP] Unpacking FMU into C:\Users\ALFOA\AppData\Local\Temp\1\fmucktmpa00344
[VERBOSE][FMIXML] Parsing XML to detect FMI standard version
[VERBOSE][FMIXML] XML specifies FMI 2.0
[INFO][FMILIB] XML specifies FMI standard version 2.0
[VERBOSE][FMILIB] Parsing model description XML
[VERBOSE][FMI2XML] Parsing XML element fmiModelDescription
[VERBOSE][FMI2XML] Parsing XML element CoSimulation
[VERBOSE][FMI2XML] Parsing XML element LogCategories
[VERBOSE][FMI2XML] Parsing XML element ModelVariables
[VERBOSE][FMI2XML] Building alias index
[VERBOSE][FMI2XML] Parsing XML element ModelStructure
[VERBOSE][FMI2XML] Parsing XML element Outputs
[VERBOSE][FMILIB] Parsing finished successfully
[INFO][FMUCHK] Model name: ROMSlave
[INFO][FMUCHK] Model GUID: 497a462e-7e10-11eb-91e7-54bf645b8fcc
[INFO][FMUCHK] Model version:
[INFO][FMUCHK] FMU kind: CoSimulation
[INFO][FMUCHK] The FMU contains:
0 constants
2 parameters
0 discrete variables
2 continuous variables
1 inputs
1 outputs
0 local variables
0 independent variables
0 calculated parameters
2 real variables
0 integer variables
0 enumeration variables
0 boolean variables
2 string variables

[INFO][FMUCHK] No input data provided. In case of simulation initial values from FMU will be used.
[INFO][FMUCHK] Printing output file header
[INFO][FMUCHK] Model identifier for CoSimulation: ROMSlave
[INFO][FMILIB] Loading 'win64' binary with 'default' platform types
[VERBOSE][FMICAPI] Loaded FMU binary from C:\Users\ALFOA\AppData\Local\Temp\1\fmucktmpa00344\binaries\win64\ROMSlave.dll
[VERBOSE][FMICAPI] Loading functions for the co-simulation interface
[VERBOSE][FMILIB] Successfully loaded all the interface functions
[INFO][FMUCHK] Version returned from CS FMU:   2.0
[VERBOSE][FMICAPI] Calling fmi2GetModelTypesPlatform
[VERBOSE][FMUCHK] allocateMemory( 1, 192) called. Returning pointer: 000001C6167D3EE0
[VERBOSE][FMUCHK] allocateMemory( 1, 160) called. Returning pointer: 000001C618A33C70
[VERBOSE][FMICAPI] Calling fmi2SetupExperiment
[VERBOSE][FMICAPI] Calling fmi2EnterInitializationMode
[VERBOSE][FMICAPI] Calling fmi2ExitInitializationMode
[INFO][FMUCHK] Initialized FMU for simulation starting at time 0
[VERBOSE][FMUCHK] Writing simulation output for start time
[VERBOSE][FMUCHK] Simulation step from time: 0 until: 0.002
[ERROR][FMUCHK] FMU logger callback does not propagate component environment to the application
        [FMU][FMU status:Fatal] Fatal py exception encountered: [doStep] PyObject_CallMethod
ImportError('\n\nIMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!\n\nImporting the numpy C-extensions failed. This error can happen for\nmany reasons, often due to issues with your setup or how NumPy was\ninstalled.\n\nWe have compiled some common reasons and troubleshooting tips at:\n\n    https://numpy.org/devdocs/user/troubleshooting-importerror.html\n\nPlease note and check the following:\n\n  * The Python version is: Python3.7 from "C:\\Users\\ALFOA\\projects\\FMUComplianceChecker\\build\\Debug\\fmuCheck.win64.exe"\n  * The NumPy version is: "1.18.5"\n\nand make sure that they are the versions you expect.\nPlease carefully study the documentation linked above for further help.\n\nOriginal error was: DLL load failed: The specified module could not be found.\n')
[FATAL][FMUCHK] Simulation loop terminated at time 0.002 since FMU returned status: Fatal
[ERROR][FMUCHK] fmiTerminateSlave returned status: Fatal
[VERBOSE][FMILIB] Releasing allocated library resources
[VERBOSE][FMILIB] Releasing FMU CAPI interface
C:\Users\ALFOA\AppData\Local\Continuum\miniconda3\envs\raven_libraries\lib\site-packages\numpy\__init__.py:140: UserWarning: mkl-service package failed to import, therefore Intel(R) MKL initialization ensuring its correct out-of-the box operation under condition when Gnu OpenMP had already been loaded by Python process is not assured. Please install mkl-service package, see http://github.com/IntelPython/mkl-service
  from . import _distributor_init
[VERBOSE][FMICAPI] Successfully unloaded FMU binary
[VERBOSE][JMPRT] Removing C:\Users\ALFOA\AppData\Local\Temp\1\fmucktmpa00344
[ERROR][FMUCHK] Memory leak: freeMemory was not called for 2 block(s) allocated by allocateMemory
FMU check summary:
FMU reported:
        1 warning(s) and error(s)
Checker reported:
        0 Warning(s)
        4 Error(s)
        1 Fatal error(s) occurred during processing

I am running on Win 64bit Running Win 10.

[alfoa]

The following Fmi2Slave derived class shows the error:

from pythonfmu import Fmi2Causality, Fmi2Slave, Fmi2Variability,  Boolean, Integer, Real, String
import os
import sys
#import numpy as np
# Prototype ROM exporter in FMU
class ROMSlave(Fmi2Slave):
    """
    RAVEN (raven.inl.gov) ROM-based Python-driven simulator
    """
    author = "Andrea Alfonsi"
    description = "ROM-based Python-driven simulator"

    def __init__(self, **kwargs):
        """
          The configuration file here represents the surrogate model that needs to be loaded
        """
        super().__init__(**kwargs)
        #import pickle
        
        # the configuration file here is a support file where the raven framework
        # and the 
        self.SES_Demand =  352321536
        self.SES_generator_P_flow = 335544320
        self.raven_path = "framework"
        self.model_path = "ROMpk.pk"
        #sys.path.append(self.raven_path)
        self.initialized = False
        self.register_variable(Real("SES_Demand", causality=Fmi2Causality.input))
        self.register_variable(Real("SES_generator_P_flow", causality=Fmi2Causality.output))
        self.register_variable(String("raven_path", causality=Fmi2Causality.parameter, variability=Fmi2Variability.tunable))
        self.register_variable(String("model_path", causality=Fmi2Causality.parameter, variability=Fmi2Variability.tunable))
        
 
    def do_step(self, current_time, step_size):
        import numpy as np
        #if not self.initialized:    
        #    # de-serialize the model
        #    self.rom = pickle.load(open(self.model_path, mode='rb'))            
        #    self.initialized = True
        #request = {"demand":np.array(self.SES_Demand)}
        #outs = self.rom.evaluate(request)
        #for var in outs:
        #    self.__dict__[var] = outs[var][0]
        self.SES_generator_P_flow = 100000000
        return True

[markaren]

Numpy is an issue with some. Did you try to follow the steps reported by numpy in your reported error message?

[markaren]

Note that numpy works on my side. I only have one Python (plain python)/numpy(pip) installation.