Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue with ModelGraph function during ONNX model synthesis #983

Open
sei-rquartiano opened this issue Mar 18, 2024 · 4 comments
Open

Issue with ModelGraph function during ONNX model synthesis #983

sei-rquartiano opened this issue Mar 18, 2024 · 4 comments
Labels

Comments

@sei-rquartiano
Copy link
Contributor

On up-to-date main branch, running from within this docker image

Attempting to synthesize hls from ONNX model. First, I ran python code from README.md to convert Keras model without error. Then I modified the script to convert from ONNX instead (see below)

import hls4ml

# Fetch a keras model from our example repository
# This will download our example model to your working directory and return an example configuration file
config = hls4ml.utils.fetch_example_model('three_layer_keras.onnx')

# You can print the configuration to see some default parameters
print(config)

# Convert it to a hls project
hls_model = hls4ml.converters.onnx_to_hls(config)

#hls_model.build(csim=False)

# Print full list of example models if you want to explore more
hls4ml.utils.fetch_example_list()

When I run the above code, I get the following output and subsequent error message during synthesis


2024-03-18 17:48:46.474669: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2024-03-18 17:48:47.242971: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2024-03-18 17:48:47.243534: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-03-18 17:48:49.843563: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Downloading example model files ...
{'OutputDir': 'my-hls-test', 'ProjectName': 'myproject', 'Backend': 'Vivado', 'Version': '1.0.0', 'Part': 'xcvu13p-flga2577-2-e', 'ClockPeriod': 5, 'IOType': 'io_parallel', 'HLSConfig': {'Model': {'Precision': 'ap_fixed<16,6>', 'ReuseFactor': 1}}, 'OnnxModel': 'three_layer_keras.onnx'}
Interpreting Model ...
Output layers:  ['Softmax']
Input shape: [None, 16]
Topology:
Layer name: _class__keras_layers_core_Dense__, layer type: Dense, current shape: [[None, 16]]
Layer name: Add, layer type: Add, current shape: [[None, 64]]
Layer name: Relu, layer type: Activation, current shape: [[None, 64]]
Layer name: _class__keras_layers_core_Dense__1, layer type: Dense, current shape: [[None, 64]]
Layer name: Add1, layer type: Add, current shape: [[None, 32]]
Layer name: Relu1, layer type: Activation, current shape: [[None, 32]]
Layer name: _class__keras_layers_core_Dense__2, layer type: Dense, current shape: [[None, 32]]
Layer name: Add2, layer type: Add, current shape: [[None, 32]]
Layer name: Relu2, layer type: Activation, current shape: [[None, 32]]
Layer name: _class__keras_layers_core_Dense__3, layer type: Dense, current shape: [[None, 32]]
Layer name: Add3, layer type: Add, current shape: [[None, 5]]
Layer name: Softmax, layer type: Softmax, current shape: [[None, 5]]
Creating HLS model
Traceback (most recent call last):
  File "/home/hls4ml-user/work/hls4ml/example_script.py", line 11, in <module>
    hls_model = hls4ml.converters.onnx_to_hls(config)
  File "/home/hls4ml-user/work/hls4ml/hls4ml/converters/onnx_to_hls.py", line 318, in onnx_to_hls
    hls_model = ModelGraph(config, reader, layer_list, input_layers, output_layers)
TypeError: ModelGraph.__init__() takes from 3 to 5 positional arguments but 6 were given

I originally encountered this error in a different and more complex project, but reproduced it in hls4ml's example code to facilitate debugging

Thank you!

@vloncar
Copy link
Contributor

vloncar commented Mar 18, 2024

The current ONNX parser is broken in many ways, it is going to be updated soon. You can try out the development version in #979

@sei-jgwohlbier
Copy link

@vloncar what is the expectation on the tests in #979 ? I get

= 58 failed, 1280 passed, 6 skipped, 649 warnings, 8 errors in 3587.97s (0:59:47) =

In particular test_qonnx.py does not pass.

(hls4ml) hls4ml-user@lx-pinkbird:~/hls4ml/test/pytest$ python -m pytest test_qonnx.py 
============================= test session starts ==============================
platform linux -- Python 3.10.14, pytest-8.1.1, pluggy-1.4.0
rootdir: /home/hls4ml-user/hls4ml
configfile: pyproject.toml
collected 9 items                                                              

test_qonnx.py EEEEEEFFF                                                  [100%]

==================================== ERRORS ====================================
___________________ ERROR at setup of test_tfc_2w2a[Vivado] ____________________

    @pytest.fixture(scope='module')
    def tfc_2w2a_model():
        '''
        Load the tiny fully-connected model
        '''
        dl_dir = test_root_path
        dl_file = str(dl_dir / "qonnx-tfc-2w2a.onnx")
        tfc_w2a2_qonnx_url = (
            "https://raw.githubusercontent.com/fastmachinelearning/"
            "QONNX_model_zoo/main/models/MNIST/Brevitas_FINN_TFC/TFC/TFC_2W2A.onnx"
        )
        urllib.request.urlretrieve(tfc_w2a2_qonnx_url, dl_file)
        assert os.path.isfile(dl_file)
        out_file = str(dl_dir / "qonnx-tfc-2w2a-clean.onnx")
    
        # cleanup
>       qonnx.util.cleanup.cleanup(dl_file, out_file=out_file)

test_qonnx.py:35: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/cleanup.py:66: in cleanup
    model = cleanup_model(model, preserve_qnt_ops=preserve_qnt_ops, override_batchsize=override_batchsize)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/cleanup.py:46: in cleanup_model
    model = model.transform(t)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/modelwrapper.py:140: in transform
    (transformed_model, model_was_changed) = transformation.apply(transformed_model)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/transformation/fold_constants.py:108: in apply
    oxe.execute_node(n, execution_context, model.graph, opset_version=opset_version)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/onnx_exec.py:87: in execute_node
    sess = rt.InferenceSession(node_model.SerializeToString())
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:419: in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <onnxruntime.capi.onnxruntime_inference_collection.InferenceSession object at 0x7ff2b2cd78b0>
providers = [], provider_options = [], disabled_optimizers = None

    def _create_inference_session(self, providers, provider_options, disabled_optimizers=None):
        available_providers = C.get_available_providers()
    
        # Tensorrt can fall back to CUDA if it's explicitly assigned. All others fall back to CPU.
        if "TensorrtExecutionProvider" in available_providers:
            if providers and any(
                provider == "CUDAExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "CUDAExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        # MIGraphX can fall back to ROCM if it's explicitly assigned. All others fall back to CPU.
        elif "MIGraphXExecutionProvider" in available_providers:
            if providers and any(
                provider == "ROCMExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "ROCMExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["ROCMExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        else:
            self._fallback_providers = ["CPUExecutionProvider"]
    
        # validate providers and provider_options before other initialization
        providers, provider_options = check_and_normalize_provider_args(
            providers, provider_options, available_providers
        )
    
        session_options = self._sess_options if self._sess_options else C.get_default_session_options()
    
        self._register_ep_custom_ops(session_options, providers, provider_options, available_providers)
    
        if self._model_path:
            sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
        else:
>           sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)
E           onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Failed to load model with error: /onnxruntime_src/onnxruntime/core/graph/model.cc:179 onnxruntime::Model::Model(onnx::ModelProto&&, const onnxruntime::PathString&, const onnxruntime::IOnnxRuntimeOpSchemaRegistryList*, const onnxruntime::logging::Logger&, const onnxruntime::ModelOptions&) Unsupported model IR version: 10, max supported IR version: 9

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:474: InvalidArgument
____________________ ERROR at setup of test_tfc_2w2a[Vitis] ____________________

    @pytest.fixture(scope='module')
    def tfc_2w2a_model():
        '''
        Load the tiny fully-connected model
        '''
        dl_dir = test_root_path
        dl_file = str(dl_dir / "qonnx-tfc-2w2a.onnx")
        tfc_w2a2_qonnx_url = (
            "https://raw.githubusercontent.com/fastmachinelearning/"
            "QONNX_model_zoo/main/models/MNIST/Brevitas_FINN_TFC/TFC/TFC_2W2A.onnx"
        )
        urllib.request.urlretrieve(tfc_w2a2_qonnx_url, dl_file)
        assert os.path.isfile(dl_file)
        out_file = str(dl_dir / "qonnx-tfc-2w2a-clean.onnx")
    
        # cleanup
>       qonnx.util.cleanup.cleanup(dl_file, out_file=out_file)

test_qonnx.py:35: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/cleanup.py:66: in cleanup
    model = cleanup_model(model, preserve_qnt_ops=preserve_qnt_ops, override_batchsize=override_batchsize)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/cleanup.py:46: in cleanup_model
    model = model.transform(t)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/modelwrapper.py:140: in transform
    (transformed_model, model_was_changed) = transformation.apply(transformed_model)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/transformation/fold_constants.py:108: in apply
    oxe.execute_node(n, execution_context, model.graph, opset_version=opset_version)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/onnx_exec.py:87: in execute_node
    sess = rt.InferenceSession(node_model.SerializeToString())
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:419: in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <onnxruntime.capi.onnxruntime_inference_collection.InferenceSession object at 0x7ff2b2cd78b0>
providers = [], provider_options = [], disabled_optimizers = None

    def _create_inference_session(self, providers, provider_options, disabled_optimizers=None):
        available_providers = C.get_available_providers()
    
        # Tensorrt can fall back to CUDA if it's explicitly assigned. All others fall back to CPU.
        if "TensorrtExecutionProvider" in available_providers:
            if providers and any(
                provider == "CUDAExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "CUDAExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        # MIGraphX can fall back to ROCM if it's explicitly assigned. All others fall back to CPU.
        elif "MIGraphXExecutionProvider" in available_providers:
            if providers and any(
                provider == "ROCMExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "ROCMExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["ROCMExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        else:
            self._fallback_providers = ["CPUExecutionProvider"]
    
        # validate providers and provider_options before other initialization
        providers, provider_options = check_and_normalize_provider_args(
            providers, provider_options, available_providers
        )
    
        session_options = self._sess_options if self._sess_options else C.get_default_session_options()
    
        self._register_ep_custom_ops(session_options, providers, provider_options, available_providers)
    
        if self._model_path:
            sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
        else:
>           sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)
E           onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Failed to load model with error: /onnxruntime_src/onnxruntime/core/graph/model.cc:179 onnxruntime::Model::Model(onnx::ModelProto&&, const onnxruntime::PathString&, const onnxruntime::IOnnxRuntimeOpSchemaRegistryList*, const onnxruntime::logging::Logger&, const onnxruntime::ModelOptions&) Unsupported model IR version: 10, max supported IR version: 9

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:474: InvalidArgument
___________________ ERROR at setup of test_tfc_2w2a[Quartus] ___________________

    @pytest.fixture(scope='module')
    def tfc_2w2a_model():
        '''
        Load the tiny fully-connected model
        '''
        dl_dir = test_root_path
        dl_file = str(dl_dir / "qonnx-tfc-2w2a.onnx")
        tfc_w2a2_qonnx_url = (
            "https://raw.githubusercontent.com/fastmachinelearning/"
            "QONNX_model_zoo/main/models/MNIST/Brevitas_FINN_TFC/TFC/TFC_2W2A.onnx"
        )
        urllib.request.urlretrieve(tfc_w2a2_qonnx_url, dl_file)
        assert os.path.isfile(dl_file)
        out_file = str(dl_dir / "qonnx-tfc-2w2a-clean.onnx")
    
        # cleanup
>       qonnx.util.cleanup.cleanup(dl_file, out_file=out_file)

test_qonnx.py:35: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/cleanup.py:66: in cleanup
    model = cleanup_model(model, preserve_qnt_ops=preserve_qnt_ops, override_batchsize=override_batchsize)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/cleanup.py:46: in cleanup_model
    model = model.transform(t)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/modelwrapper.py:140: in transform
    (transformed_model, model_was_changed) = transformation.apply(transformed_model)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/transformation/fold_constants.py:108: in apply
    oxe.execute_node(n, execution_context, model.graph, opset_version=opset_version)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/onnx_exec.py:87: in execute_node
    sess = rt.InferenceSession(node_model.SerializeToString())
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:419: in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <onnxruntime.capi.onnxruntime_inference_collection.InferenceSession object at 0x7ff2b2cd78b0>
providers = [], provider_options = [], disabled_optimizers = None

    def _create_inference_session(self, providers, provider_options, disabled_optimizers=None):
        available_providers = C.get_available_providers()
    
        # Tensorrt can fall back to CUDA if it's explicitly assigned. All others fall back to CPU.
        if "TensorrtExecutionProvider" in available_providers:
            if providers and any(
                provider == "CUDAExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "CUDAExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        # MIGraphX can fall back to ROCM if it's explicitly assigned. All others fall back to CPU.
        elif "MIGraphXExecutionProvider" in available_providers:
            if providers and any(
                provider == "ROCMExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "ROCMExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["ROCMExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        else:
            self._fallback_providers = ["CPUExecutionProvider"]
    
        # validate providers and provider_options before other initialization
        providers, provider_options = check_and_normalize_provider_args(
            providers, provider_options, available_providers
        )
    
        session_options = self._sess_options if self._sess_options else C.get_default_session_options()
    
        self._register_ep_custom_ops(session_options, providers, provider_options, available_providers)
    
        if self._model_path:
            sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
        else:
>           sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)
E           onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Failed to load model with error: /onnxruntime_src/onnxruntime/core/graph/model.cc:179 onnxruntime::Model::Model(onnx::ModelProto&&, const onnxruntime::PathString&, const onnxruntime::IOnnxRuntimeOpSchemaRegistryList*, const onnxruntime::logging::Logger&, const onnxruntime::ModelOptions&) Unsupported model IR version: 10, max supported IR version: 9

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:474: InvalidArgument
___________________ ERROR at setup of test_cnv_2w2a[Vivado] ____________________

    @pytest.fixture(scope='module')
    def cnv_2w2a_model():
        '''
        Load the small convolution model
        '''
        dl_dir = test_root_path
        dl_file = str(dl_dir / "qonnx-cnv-2w2a.onnx")
        cnv_w2a2_qonnx_url = (
            "https://raw.githubusercontent.com/fastmachinelearning/"
            "QONNX_model_zoo/main/models/CIFAR10/Brevitas_FINN_CNV/CNV_2W2A.onnx"
        )
        urllib.request.urlretrieve(cnv_w2a2_qonnx_url, dl_file)
        assert os.path.isfile(dl_file)
        out_clean = str(dl_dir / "qonnx-cnv-2w2a-clean.onnx")
        out_chanlast = str(dl_dir / "qonnx-cnv-2w2a-clean-channels-last.onnx")
        out_file = str(dl_dir / "qonnx-cnv-2w2a-clean-channels-last-clean.onnx")
    
        # cleanup
>       qonnx.util.cleanup.cleanup(dl_file, out_file=out_clean)

test_qonnx.py:58: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/cleanup.py:66: in cleanup
    model = cleanup_model(model, preserve_qnt_ops=preserve_qnt_ops, override_batchsize=override_batchsize)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/cleanup.py:46: in cleanup_model
    model = model.transform(t)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/modelwrapper.py:140: in transform
    (transformed_model, model_was_changed) = transformation.apply(transformed_model)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/transformation/fold_constants.py:108: in apply
    oxe.execute_node(n, execution_context, model.graph, opset_version=opset_version)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/onnx_exec.py:87: in execute_node
    sess = rt.InferenceSession(node_model.SerializeToString())
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:419: in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <onnxruntime.capi.onnxruntime_inference_collection.InferenceSession object at 0x7ff2b0b907f0>
providers = [], provider_options = [], disabled_optimizers = None

    def _create_inference_session(self, providers, provider_options, disabled_optimizers=None):
        available_providers = C.get_available_providers()
    
        # Tensorrt can fall back to CUDA if it's explicitly assigned. All others fall back to CPU.
        if "TensorrtExecutionProvider" in available_providers:
            if providers and any(
                provider == "CUDAExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "CUDAExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        # MIGraphX can fall back to ROCM if it's explicitly assigned. All others fall back to CPU.
        elif "MIGraphXExecutionProvider" in available_providers:
            if providers and any(
                provider == "ROCMExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "ROCMExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["ROCMExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        else:
            self._fallback_providers = ["CPUExecutionProvider"]
    
        # validate providers and provider_options before other initialization
        providers, provider_options = check_and_normalize_provider_args(
            providers, provider_options, available_providers
        )
    
        session_options = self._sess_options if self._sess_options else C.get_default_session_options()
    
        self._register_ep_custom_ops(session_options, providers, provider_options, available_providers)
    
        if self._model_path:
            sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
        else:
>           sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)
E           onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Failed to load model with error: /onnxruntime_src/onnxruntime/core/graph/model.cc:179 onnxruntime::Model::Model(onnx::ModelProto&&, const onnxruntime::PathString&, const onnxruntime::IOnnxRuntimeOpSchemaRegistryList*, const onnxruntime::logging::Logger&, const onnxruntime::ModelOptions&) Unsupported model IR version: 10, max supported IR version: 9

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:474: InvalidArgument
____________________ ERROR at setup of test_cnv_2w2a[Vitis] ____________________

    @pytest.fixture(scope='module')
    def cnv_2w2a_model():
        '''
        Load the small convolution model
        '''
        dl_dir = test_root_path
        dl_file = str(dl_dir / "qonnx-cnv-2w2a.onnx")
        cnv_w2a2_qonnx_url = (
            "https://raw.githubusercontent.com/fastmachinelearning/"
            "QONNX_model_zoo/main/models/CIFAR10/Brevitas_FINN_CNV/CNV_2W2A.onnx"
        )
        urllib.request.urlretrieve(cnv_w2a2_qonnx_url, dl_file)
        assert os.path.isfile(dl_file)
        out_clean = str(dl_dir / "qonnx-cnv-2w2a-clean.onnx")
        out_chanlast = str(dl_dir / "qonnx-cnv-2w2a-clean-channels-last.onnx")
        out_file = str(dl_dir / "qonnx-cnv-2w2a-clean-channels-last-clean.onnx")
    
        # cleanup
>       qonnx.util.cleanup.cleanup(dl_file, out_file=out_clean)

test_qonnx.py:58: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/cleanup.py:66: in cleanup
    model = cleanup_model(model, preserve_qnt_ops=preserve_qnt_ops, override_batchsize=override_batchsize)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/cleanup.py:46: in cleanup_model
    model = model.transform(t)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/modelwrapper.py:140: in transform
    (transformed_model, model_was_changed) = transformation.apply(transformed_model)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/transformation/fold_constants.py:108: in apply
    oxe.execute_node(n, execution_context, model.graph, opset_version=opset_version)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/onnx_exec.py:87: in execute_node
    sess = rt.InferenceSession(node_model.SerializeToString())
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:419: in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <onnxruntime.capi.onnxruntime_inference_collection.InferenceSession object at 0x7ff2b0b907f0>
providers = [], provider_options = [], disabled_optimizers = None

    def _create_inference_session(self, providers, provider_options, disabled_optimizers=None):
        available_providers = C.get_available_providers()
    
        # Tensorrt can fall back to CUDA if it's explicitly assigned. All others fall back to CPU.
        if "TensorrtExecutionProvider" in available_providers:
            if providers and any(
                provider == "CUDAExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "CUDAExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        # MIGraphX can fall back to ROCM if it's explicitly assigned. All others fall back to CPU.
        elif "MIGraphXExecutionProvider" in available_providers:
            if providers and any(
                provider == "ROCMExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "ROCMExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["ROCMExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        else:
            self._fallback_providers = ["CPUExecutionProvider"]
    
        # validate providers and provider_options before other initialization
        providers, provider_options = check_and_normalize_provider_args(
            providers, provider_options, available_providers
        )
    
        session_options = self._sess_options if self._sess_options else C.get_default_session_options()
    
        self._register_ep_custom_ops(session_options, providers, provider_options, available_providers)
    
        if self._model_path:
            sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
        else:
>           sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)
E           onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Failed to load model with error: /onnxruntime_src/onnxruntime/core/graph/model.cc:179 onnxruntime::Model::Model(onnx::ModelProto&&, const onnxruntime::PathString&, const onnxruntime::IOnnxRuntimeOpSchemaRegistryList*, const onnxruntime::logging::Logger&, const onnxruntime::ModelOptions&) Unsupported model IR version: 10, max supported IR version: 9

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:474: InvalidArgument
___________________ ERROR at setup of test_cnv_2w2a[Quartus] ___________________

    @pytest.fixture(scope='module')
    def cnv_2w2a_model():
        '''
        Load the small convolution model
        '''
        dl_dir = test_root_path
        dl_file = str(dl_dir / "qonnx-cnv-2w2a.onnx")
        cnv_w2a2_qonnx_url = (
            "https://raw.githubusercontent.com/fastmachinelearning/"
            "QONNX_model_zoo/main/models/CIFAR10/Brevitas_FINN_CNV/CNV_2W2A.onnx"
        )
        urllib.request.urlretrieve(cnv_w2a2_qonnx_url, dl_file)
        assert os.path.isfile(dl_file)
        out_clean = str(dl_dir / "qonnx-cnv-2w2a-clean.onnx")
        out_chanlast = str(dl_dir / "qonnx-cnv-2w2a-clean-channels-last.onnx")
        out_file = str(dl_dir / "qonnx-cnv-2w2a-clean-channels-last-clean.onnx")
    
        # cleanup
>       qonnx.util.cleanup.cleanup(dl_file, out_file=out_clean)

test_qonnx.py:58: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/cleanup.py:66: in cleanup
    model = cleanup_model(model, preserve_qnt_ops=preserve_qnt_ops, override_batchsize=override_batchsize)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/cleanup.py:46: in cleanup_model
    model = model.transform(t)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/modelwrapper.py:140: in transform
    (transformed_model, model_was_changed) = transformation.apply(transformed_model)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/transformation/fold_constants.py:108: in apply
    oxe.execute_node(n, execution_context, model.graph, opset_version=opset_version)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/onnx_exec.py:87: in execute_node
    sess = rt.InferenceSession(node_model.SerializeToString())
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:419: in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <onnxruntime.capi.onnxruntime_inference_collection.InferenceSession object at 0x7ff2b0b907f0>
providers = [], provider_options = [], disabled_optimizers = None

    def _create_inference_session(self, providers, provider_options, disabled_optimizers=None):
        available_providers = C.get_available_providers()
    
        # Tensorrt can fall back to CUDA if it's explicitly assigned. All others fall back to CPU.
        if "TensorrtExecutionProvider" in available_providers:
            if providers and any(
                provider == "CUDAExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "CUDAExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        # MIGraphX can fall back to ROCM if it's explicitly assigned. All others fall back to CPU.
        elif "MIGraphXExecutionProvider" in available_providers:
            if providers and any(
                provider == "ROCMExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "ROCMExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["ROCMExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        else:
            self._fallback_providers = ["CPUExecutionProvider"]
    
        # validate providers and provider_options before other initialization
        providers, provider_options = check_and_normalize_provider_args(
            providers, provider_options, available_providers
        )
    
        session_options = self._sess_options if self._sess_options else C.get_default_session_options()
    
        self._register_ep_custom_ops(session_options, providers, provider_options, available_providers)
    
        if self._model_path:
            sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
        else:
>           sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)
E           onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Failed to load model with error: /onnxruntime_src/onnxruntime/core/graph/model.cc:179 onnxruntime::Model::Model(onnx::ModelProto&&, const onnxruntime::PathString&, const onnxruntime::IOnnxRuntimeOpSchemaRegistryList*, const onnxruntime::logging::Logger&, const onnxruntime::ModelOptions&) Unsupported model IR version: 10, max supported IR version: 9

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:474: InvalidArgument
=================================== FAILURES ===================================
___________________________ test_jet_tagging[Vivado] ___________________________

jettagging_model = <qonnx.core.modelwrapper.ModelWrapper object at 0x7ff2b09deb30>
backend = 'Vivado'

    @pytest.mark.parametrize('backend', ['Vivado', 'Vitis', 'Quartus'])
    def test_jet_tagging(jettagging_model, backend):
        model = jettagging_model
    
        # Execute QONNX model inference
        # TODO make the test bigger
        ishape = (1, 16)
        X = np.random.uniform(low=-1, high=+1, size=np.prod(ishape)).reshape(ishape)
        X = (np.round(X * 2**16) * 2**-16).astype(np.float32)
        idict = {model.graph.input[0].name: X}
>       y_qonnx = oxe.execute_onnx(model, idict)[model.graph.output[0].name]

test_qonnx.py:147: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/onnx_exec.py:179: in execute_onnx
    execute_node(node, execution_context, graph, return_full_exec_context, opset_version)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/onnx_exec.py:87: in execute_node
    sess = rt.InferenceSession(node_model.SerializeToString())
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:419: in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <onnxruntime.capi.onnxruntime_inference_collection.InferenceSession object at 0x7ff2b09dde70>
providers = [], provider_options = [], disabled_optimizers = None

    def _create_inference_session(self, providers, provider_options, disabled_optimizers=None):
        available_providers = C.get_available_providers()
    
        # Tensorrt can fall back to CUDA if it's explicitly assigned. All others fall back to CPU.
        if "TensorrtExecutionProvider" in available_providers:
            if providers and any(
                provider == "CUDAExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "CUDAExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        # MIGraphX can fall back to ROCM if it's explicitly assigned. All others fall back to CPU.
        elif "MIGraphXExecutionProvider" in available_providers:
            if providers and any(
                provider == "ROCMExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "ROCMExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["ROCMExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        else:
            self._fallback_providers = ["CPUExecutionProvider"]
    
        # validate providers and provider_options before other initialization
        providers, provider_options = check_and_normalize_provider_args(
            providers, provider_options, available_providers
        )
    
        session_options = self._sess_options if self._sess_options else C.get_default_session_options()
    
        self._register_ep_custom_ops(session_options, providers, provider_options, available_providers)
    
        if self._model_path:
            sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
        else:
>           sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)
E           onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Failed to load model with error: /onnxruntime_src/onnxruntime/core/graph/model.cc:179 onnxruntime::Model::Model(onnx::ModelProto&&, const onnxruntime::PathString&, const onnxruntime::IOnnxRuntimeOpSchemaRegistryList*, const onnxruntime::logging::Logger&, const onnxruntime::ModelOptions&) Unsupported model IR version: 10, max supported IR version: 9

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:474: InvalidArgument
___________________________ test_jet_tagging[Vitis] ____________________________

jettagging_model = <qonnx.core.modelwrapper.ModelWrapper object at 0x7ff2b09deb30>
backend = 'Vitis'

    @pytest.mark.parametrize('backend', ['Vivado', 'Vitis', 'Quartus'])
    def test_jet_tagging(jettagging_model, backend):
        model = jettagging_model
    
        # Execute QONNX model inference
        # TODO make the test bigger
        ishape = (1, 16)
        X = np.random.uniform(low=-1, high=+1, size=np.prod(ishape)).reshape(ishape)
        X = (np.round(X * 2**16) * 2**-16).astype(np.float32)
        idict = {model.graph.input[0].name: X}
>       y_qonnx = oxe.execute_onnx(model, idict)[model.graph.output[0].name]

test_qonnx.py:147: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/onnx_exec.py:179: in execute_onnx
    execute_node(node, execution_context, graph, return_full_exec_context, opset_version)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/onnx_exec.py:87: in execute_node
    sess = rt.InferenceSession(node_model.SerializeToString())
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:419: in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <onnxruntime.capi.onnxruntime_inference_collection.InferenceSession object at 0x7ff2b0c20c40>
providers = [], provider_options = [], disabled_optimizers = None

    def _create_inference_session(self, providers, provider_options, disabled_optimizers=None):
        available_providers = C.get_available_providers()
    
        # Tensorrt can fall back to CUDA if it's explicitly assigned. All others fall back to CPU.
        if "TensorrtExecutionProvider" in available_providers:
            if providers and any(
                provider == "CUDAExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "CUDAExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        # MIGraphX can fall back to ROCM if it's explicitly assigned. All others fall back to CPU.
        elif "MIGraphXExecutionProvider" in available_providers:
            if providers and any(
                provider == "ROCMExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "ROCMExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["ROCMExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        else:
            self._fallback_providers = ["CPUExecutionProvider"]
    
        # validate providers and provider_options before other initialization
        providers, provider_options = check_and_normalize_provider_args(
            providers, provider_options, available_providers
        )
    
        session_options = self._sess_options if self._sess_options else C.get_default_session_options()
    
        self._register_ep_custom_ops(session_options, providers, provider_options, available_providers)
    
        if self._model_path:
            sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
        else:
>           sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)
E           onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Failed to load model with error: /onnxruntime_src/onnxruntime/core/graph/model.cc:179 onnxruntime::Model::Model(onnx::ModelProto&&, const onnxruntime::PathString&, const onnxruntime::IOnnxRuntimeOpSchemaRegistryList*, const onnxruntime::logging::Logger&, const onnxruntime::ModelOptions&) Unsupported model IR version: 10, max supported IR version: 9

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:474: InvalidArgument
__________________________ test_jet_tagging[Quartus] ___________________________

jettagging_model = <qonnx.core.modelwrapper.ModelWrapper object at 0x7ff2b09deb30>
backend = 'Quartus'

    @pytest.mark.parametrize('backend', ['Vivado', 'Vitis', 'Quartus'])
    def test_jet_tagging(jettagging_model, backend):
        model = jettagging_model
    
        # Execute QONNX model inference
        # TODO make the test bigger
        ishape = (1, 16)
        X = np.random.uniform(low=-1, high=+1, size=np.prod(ishape)).reshape(ishape)
        X = (np.round(X * 2**16) * 2**-16).astype(np.float32)
        idict = {model.graph.input[0].name: X}
>       y_qonnx = oxe.execute_onnx(model, idict)[model.graph.output[0].name]

test_qonnx.py:147: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/onnx_exec.py:179: in execute_onnx
    execute_node(node, execution_context, graph, return_full_exec_context, opset_version)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/onnx_exec.py:87: in execute_node
    sess = rt.InferenceSession(node_model.SerializeToString())
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:419: in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <onnxruntime.capi.onnxruntime_inference_collection.InferenceSession object at 0x7ff2b09fc310>
providers = [], provider_options = [], disabled_optimizers = None

    def _create_inference_session(self, providers, provider_options, disabled_optimizers=None):
        available_providers = C.get_available_providers()
    
        # Tensorrt can fall back to CUDA if it's explicitly assigned. All others fall back to CPU.
        if "TensorrtExecutionProvider" in available_providers:
            if providers and any(
                provider == "CUDAExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "CUDAExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        # MIGraphX can fall back to ROCM if it's explicitly assigned. All others fall back to CPU.
        elif "MIGraphXExecutionProvider" in available_providers:
            if providers and any(
                provider == "ROCMExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "ROCMExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["ROCMExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        else:
            self._fallback_providers = ["CPUExecutionProvider"]
    
        # validate providers and provider_options before other initialization
        providers, provider_options = check_and_normalize_provider_args(
            providers, provider_options, available_providers
        )
    
        session_options = self._sess_options if self._sess_options else C.get_default_session_options()
    
        self._register_ep_custom_ops(session_options, providers, provider_options, available_providers)
    
        if self._model_path:
            sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
        else:
>           sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)
E           onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Failed to load model with error: /onnxruntime_src/onnxruntime/core/graph/model.cc:179 onnxruntime::Model::Model(onnx::ModelProto&&, const onnxruntime::PathString&, const onnxruntime::IOnnxRuntimeOpSchemaRegistryList*, const onnxruntime::logging::Logger&, const onnxruntime::ModelOptions&) Unsupported model IR version: 10, max supported IR version: 9

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:474: InvalidArgument
=============================== warnings summary ===============================
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/tensorflow_model_optimization/__init__.py:65
  /home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/tensorflow_model_optimization/__init__.py:65: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    if (distutils.version.LooseVersion(tf.version.VERSION) <

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/tensorflow_model_optimization/__init__.py:66
  /home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/tensorflow_model_optimization/__init__.py:66: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    distutils.version.LooseVersion(required_tensorflow_version)):

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/calmjs/parse/utils.py:13
  /home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/calmjs/parse/utils.py:13: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    from pkg_resources import working_set

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/pkg_resources/__init__.py:2871
  /home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/pkg_resources/__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('calmjs')`.
  Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
    declare_namespace(pkg)

test/pytest/test_qonnx.py: 498 warnings
  /home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/onnx.py:40: DeprecationWarning: `mapping.TENSOR_TYPE_TO_NP_TYPE` is now deprecated and will be removed in a future release.To silence this warning, please use `helper.tensor_dtype_to_np_dtype` instead.
    return np.zeros(dims, dtype=onnx.mapping.TENSOR_TYPE_TO_NP_TYPE[vi.type.tensor_type.elem_type])

test/pytest/test_qonnx.py::test_jet_tagging[Vivado]
  /home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/modelwrapper.py:93: UserWarning: Some old-style domain attributes were automatically converted to new-style,
                  i.e. domain=finn to domain=qonnx.custom_op.<general|fpgadataflow|...>
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED test_qonnx.py::test_jet_tagging[Vivado] - onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeEr...
FAILED test_qonnx.py::test_jet_tagging[Vitis] - onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeEr...
FAILED test_qonnx.py::test_jet_tagging[Quartus] - onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeEr...
ERROR test_qonnx.py::test_tfc_2w2a[Vivado] - onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeEr...
ERROR test_qonnx.py::test_tfc_2w2a[Vitis] - onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeEr...
ERROR test_qonnx.py::test_tfc_2w2a[Quartus] - onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeEr...
ERROR test_qonnx.py::test_cnv_2w2a[Vivado] - onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeEr...
ERROR test_qonnx.py::test_cnv_2w2a[Vitis] - onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeEr...
ERROR test_qonnx.py::test_cnv_2w2a[Quartus] - onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeEr...
================== 3 failed, 503 warnings, 6 errors in 5.61s ===================

@sei-jgwohlbier
Copy link

Installing onnx==1.15 and onnxruntime==1.17 seems to have cleared those onnxruntime errors.

@sei-jgwohlbier
Copy link

After clearing that issue, I do get one failure in test_qonnx.py, seen below.

(hls4ml) hls4ml-user@lx-pinkbird:~/hls4ml/test/pytest$ python -m pytest test_qonnx.py 
============================= test session starts ==============================
platform linux -- Python 3.10.14, pytest-8.1.1, pluggy-1.4.0
rootdir: /home/hls4ml-user/hls4ml
configfile: pyproject.toml
collected 9 items                                                              

test_qonnx.py .....F...                                                  [100%]

=================================== FAILURES ===================================
____________________________ test_cnv_2w2a[Quartus] ____________________________

cnv_2w2a_model = <qonnx.core.modelwrapper.ModelWrapper object at 0x7fce0ad28c70>
backend = 'Quartus'

    @pytest.mark.parametrize('backend', ['Vivado', 'Vitis', 'Quartus'])
    def test_cnv_2w2a(cnv_2w2a_model, backend):
        """
        This tests a convolution model. Note:  the batch normalizations weights not quantized, so it
        is difficult to make this match perfectly. It is also a slow test.
        """
        model = cnv_2w2a_model
    
        ishape = (1, 32, 32, 3)
        X = np.random.uniform(low=-1, high=+1, size=np.prod(ishape)).reshape(ishape)
        X = (np.round(X * 2**6) * 2**-6).astype(np.float32)
        idict = {model.graph.input[0].name: X}
        y_qonnx = oxe.execute_onnx(model, idict)[model.graph.output[0].name]
    
        # Convert QONNX model, compile, and run inference
        config = hls4ml.utils.config_from_onnx_model(model, backend=backend, default_precision='fixed<32,6>')
        hls_model = hls4ml.converters.convert_from_onnx_model(
            model,
            output_dir=str(test_root_path / f'hls4mlprj_qonnx_cnv-2w2a_{backend}'),
            io_type='io_stream',
            backend=backend,
            hls_config=config,
        )
>       hls_model.compile()

test_qonnx.py:131: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/hls4ml/model/graph.py:672: in compile
    self._compile()
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/hls4ml/model/graph.py:691: in _compile
    self._top_function_lib = ctypes.cdll.LoadLibrary(lib_name)
../../../miniconda3/envs/hls4ml/lib/python3.10/ctypes/__init__.py:452: in LoadLibrary
    return self._dlltype(name)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <CDLL '/home/hls4ml-user/hls4ml/test/pytest/hls4mlprj_qonnx_cnv-2w2a_Quartus/firmware/myproject-D9B7bA7c.so', handle 0 at 0x7fce0a534f40>
name = '/home/hls4ml-user/hls4ml/test/pytest/hls4mlprj_qonnx_cnv-2w2a_Quartus/firmware/myproject-D9B7bA7c.so'
mode = 0, handle = None, use_errno = False, use_last_error = False
winmode = None

    def __init__(self, name, mode=DEFAULT_MODE, handle=None,
                 use_errno=False,
                 use_last_error=False,
                 winmode=None):
        self._name = name
        flags = self._func_flags_
        if use_errno:
            flags |= _FUNCFLAG_USE_ERRNO
        if use_last_error:
            flags |= _FUNCFLAG_USE_LASTERROR
        if _sys.platform.startswith("aix"):
            """When the name contains ".a(" and ends with ")",
               e.g., "libFOO.a(libFOO.so)" - this is taken to be an
               archive(member) syntax for dlopen(), and the mode is adjusted.
               Otherwise, name is presented to dlopen() as a file argument.
            """
            if name and name.endswith(")") and ".a(" in name:
                mode |= ( _os.RTLD_MEMBER | _os.RTLD_NOW )
        if _os.name == "nt":
            if winmode is not None:
                mode = winmode
            else:
                import nt
                mode = nt._LOAD_LIBRARY_SEARCH_DEFAULT_DIRS
                if '/' in name or '\\' in name:
                    self._name = nt._getfullpathname(self._name)
                    mode |= nt._LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR
    
        class _FuncPtr(_CFuncPtr):
            _flags_ = flags
            _restype_ = self._func_restype_
        self._FuncPtr = _FuncPtr
    
        if handle is None:
>           self._handle = _dlopen(self._name, mode)
E           OSError: /home/hls4ml-user/hls4ml/test/pytest/hls4mlprj_qonnx_cnv-2w2a_Quartus/firmware/myproject-D9B7bA7c.so: cannot open shared object file: No such file or directory

../../../miniconda3/envs/hls4ml/lib/python3.10/ctypes/__init__.py:374: OSError
----------------------------- Captured stdout call -----------------------------
Output layers:  ['Add_0']
Input shape: [32, 32, 3]
Topology:
Layer name: Quant_0, layer type: Quant, current shape: [[64, 3, 3, 3]]
Layer name: Quant_1, layer type: Quant, current shape: [[64, 3, 3, 64]]
Layer name: Quant_2, layer type: Quant, current shape: [[128, 3, 3, 64]]
Layer name: Quant_3, layer type: Quant, current shape: [[128, 3, 3, 128]]
Layer name: Quant_4, layer type: Quant, current shape: [[256, 3, 3, 128]]
Layer name: Quant_5, layer type: Quant, current shape: [[256, 3, 3, 256]]
Layer name: Quant_6, layer type: Quant, current shape: [[256, 512]]
Layer name: Quant_7, layer type: Quant, current shape: [[512, 512]]
Layer name: Quant_8, layer type: Quant, current shape: [[512, 10]]
Layer name: Mul_0, layer type: Merge, current shape: [[1, 32, 32, 3]]
Layer name: Sub_0, layer type: Merge, current shape: [[1, 32, 32, 3], [1]]
Layer name: Quant_9, layer type: Quant, current shape: [[1, 32, 32, 3]]
Layer name: Conv_0, layer type: Conv, current shape: [[1, 32, 32, 3], [64, 3, 3, 3]]
Layer name: BatchNormalization_0, layer type: BatchNormOnnx, current shape: [[1, 30, 30, 64], [64], [64], [64], [64]]
Layer name: Quant_10, layer type: Quant, current shape: [[1, 30, 30, 64]]
Layer name: Conv_1, layer type: Conv, current shape: [[1, 30, 30, 64], [64, 3, 3, 64]]
Layer name: BatchNormalization_1, layer type: BatchNormOnnx, current shape: [[1, 28, 28, 64], [64], [64], [64], [64]]
Layer name: Quant_11, layer type: Quant, current shape: [[1, 28, 28, 64]]
Layer name: MaxPool_0, layer type: MaxPooling2D, current shape: [[1, 28, 28, 64]]
Layer name: Conv_2, layer type: Conv, current shape: [[1, 14, 14, 64], [128, 3, 3, 64]]
Layer name: BatchNormalization_2, layer type: BatchNormOnnx, current shape: [[1, 12, 12, 128], [128], [128], [128], [128]]
Layer name: Quant_12, layer type: Quant, current shape: [[1, 12, 12, 128]]
Layer name: Conv_3, layer type: Conv, current shape: [[1, 12, 12, 128], [128, 3, 3, 128]]
Layer name: BatchNormalization_3, layer type: BatchNormOnnx, current shape: [[1, 10, 10, 128], [128], [128], [128], [128]]
Layer name: Quant_13, layer type: Quant, current shape: [[1, 10, 10, 128]]
Layer name: MaxPool_1, layer type: MaxPooling2D, current shape: [[1, 10, 10, 128]]
Layer name: Conv_4, layer type: Conv, current shape: [[1, 5, 5, 128], [256, 3, 3, 128]]
Layer name: BatchNormalization_4, layer type: BatchNormOnnx, current shape: [[1, 3, 3, 256], [256], [256], [256], [256]]
Layer name: Quant_14, layer type: Quant, current shape: [[1, 3, 3, 256]]
Layer name: Conv_5, layer type: Conv, current shape: [[1, 3, 3, 256], [256, 3, 3, 256]]
Layer name: BatchNormalization_5, layer type: BatchNormOnnx, current shape: [[1, 1, 1, 256], [256], [256], [256], [256]]
Layer name: Quant_15, layer type: Quant, current shape: [[1, 1, 1, 256]]
Layer name: Flatten_0, layer type: Reshape, current shape: [[1, 1, 1, 256]]
Layer name: MatMul_0, layer type: MatMul, current shape: [[1, 256], [256, 512]]
Layer name: BatchNormalization_6, layer type: BatchNormOnnx, current shape: [[1, 512], [512], [512], [512], [512]]
Layer name: Quant_16, layer type: Quant, current shape: [[1, 512]]
Layer name: MatMul_1, layer type: MatMul, current shape: [[1, 512], [512, 512]]
Layer name: BatchNormalization_7, layer type: BatchNormOnnx, current shape: [[1, 512], [512], [512], [512], [512]]
Layer name: Quant_17, layer type: Quant, current shape: [[1, 512]]
Layer name: MatMul_2, layer type: MatMul, current shape: [[1, 512], [512, 10]]
Layer name: Sub_1, layer type: Merge, current shape: [[1, 10], [1]]
Layer name: Div_0, layer type: Merge, current shape: [[1, 10], [1]]
Layer name: Mul_1, layer type: Merge, current shape: [[1, 10], [1]]
Layer name: Add_0, layer type: Merge, current shape: [[1, 10], [1]]
Interpreting Model ...
Output layers:  ['Add_0']
Input shape: [32, 32, 3]
Topology:
Layer name: Quant_0, layer type: Quant, current shape: [[64, 3, 3, 3]]
Layer name: Quant_1, layer type: Quant, current shape: [[64, 3, 3, 64]]
Layer name: Quant_2, layer type: Quant, current shape: [[128, 3, 3, 64]]
Layer name: Quant_3, layer type: Quant, current shape: [[128, 3, 3, 128]]
Layer name: Quant_4, layer type: Quant, current shape: [[256, 3, 3, 128]]
Layer name: Quant_5, layer type: Quant, current shape: [[256, 3, 3, 256]]
Layer name: Quant_6, layer type: Quant, current shape: [[256, 512]]
Layer name: Quant_7, layer type: Quant, current shape: [[512, 512]]
Layer name: Quant_8, layer type: Quant, current shape: [[512, 10]]
Layer name: Mul_0, layer type: Merge, current shape: [[1, 32, 32, 3]]
Layer name: Sub_0, layer type: Merge, current shape: [[1, 32, 32, 3], [1]]
Layer name: Quant_9, layer type: Quant, current shape: [[1, 32, 32, 3]]
Layer name: Conv_0, layer type: Conv, current shape: [[1, 32, 32, 3], [64, 3, 3, 3]]
Layer name: BatchNormalization_0, layer type: BatchNormOnnx, current shape: [[1, 30, 30, 64], [64], [64], [64], [64]]
Layer name: Quant_10, layer type: Quant, current shape: [[1, 30, 30, 64]]
Layer name: Conv_1, layer type: Conv, current shape: [[1, 30, 30, 64], [64, 3, 3, 64]]
Layer name: BatchNormalization_1, layer type: BatchNormOnnx, current shape: [[1, 28, 28, 64], [64], [64], [64], [64]]
Layer name: Quant_11, layer type: Quant, current shape: [[1, 28, 28, 64]]
Layer name: MaxPool_0, layer type: MaxPooling2D, current shape: [[1, 28, 28, 64]]
Layer name: Conv_2, layer type: Conv, current shape: [[1, 14, 14, 64], [128, 3, 3, 64]]
Layer name: BatchNormalization_2, layer type: BatchNormOnnx, current shape: [[1, 12, 12, 128], [128], [128], [128], [128]]
Layer name: Quant_12, layer type: Quant, current shape: [[1, 12, 12, 128]]
Layer name: Conv_3, layer type: Conv, current shape: [[1, 12, 12, 128], [128, 3, 3, 128]]
Layer name: BatchNormalization_3, layer type: BatchNormOnnx, current shape: [[1, 10, 10, 128], [128], [128], [128], [128]]
Layer name: Quant_13, layer type: Quant, current shape: [[1, 10, 10, 128]]
Layer name: MaxPool_1, layer type: MaxPooling2D, current shape: [[1, 10, 10, 128]]
Layer name: Conv_4, layer type: Conv, current shape: [[1, 5, 5, 128], [256, 3, 3, 128]]
Layer name: BatchNormalization_4, layer type: BatchNormOnnx, current shape: [[1, 3, 3, 256], [256], [256], [256], [256]]
Layer name: Quant_14, layer type: Quant, current shape: [[1, 3, 3, 256]]
Layer name: Conv_5, layer type: Conv, current shape: [[1, 3, 3, 256], [256, 3, 3, 256]]
Layer name: BatchNormalization_5, layer type: BatchNormOnnx, current shape: [[1, 1, 1, 256], [256], [256], [256], [256]]
Layer name: Quant_15, layer type: Quant, current shape: [[1, 1, 1, 256]]
Layer name: Flatten_0, layer type: Reshape, current shape: [[1, 1, 1, 256]]
Layer name: MatMul_0, layer type: MatMul, current shape: [[1, 256], [256, 512]]
Layer name: BatchNormalization_6, layer type: BatchNormOnnx, current shape: [[1, 512], [512], [512], [512], [512]]
Layer name: Quant_16, layer type: Quant, current shape: [[1, 512]]
Layer name: MatMul_1, layer type: MatMul, current shape: [[1, 512], [512, 512]]
Layer name: BatchNormalization_7, layer type: BatchNormOnnx, current shape: [[1, 512], [512], [512], [512], [512]]
Layer name: Quant_17, layer type: Quant, current shape: [[1, 512]]
Layer name: MatMul_2, layer type: MatMul, current shape: [[1, 512], [512, 10]]
Layer name: Sub_1, layer type: Merge, current shape: [[1, 10], [1]]
Layer name: Div_0, layer type: Merge, current shape: [[1, 10], [1]]
Layer name: Mul_1, layer type: Merge, current shape: [[1, 10], [1]]
Layer name: Add_0, layer type: Merge, current shape: [[1, 10], [1]]
Creating HLS model
Writing HLS project
Done
----------------------------- Captured stderr call -----------------------------
g++: internal compiler error: Segmentation fault (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
g++: error: myproject.o: No such file or directory
=============================== warnings summary ===============================
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/tensorflow_model_optimization/__init__.py:65
  /home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/tensorflow_model_optimization/__init__.py:65: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    if (distutils.version.LooseVersion(tf.version.VERSION) <

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/tensorflow_model_optimization/__init__.py:66
  /home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/tensorflow_model_optimization/__init__.py:66: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    distutils.version.LooseVersion(required_tensorflow_version)):

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/calmjs/parse/utils.py:13
  /home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/calmjs/parse/utils.py:13: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    from pkg_resources import working_set

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/pkg_resources/__init__.py:2871
  /home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/pkg_resources/__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('calmjs')`.
  Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
    declare_namespace(pkg)

test/pytest/test_qonnx.py: 2101 warnings
  /home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/onnx.py:40: DeprecationWarning: `mapping.TENSOR_TYPE_TO_NP_TYPE` is now deprecated and will be removed in a future release.To silence this warning, please use `helper.tensor_dtype_to_np_dtype` instead.
    return np.zeros(dims, dtype=onnx.mapping.TENSOR_TYPE_TO_NP_TYPE[vi.type.tensor_type.elem_type])

test/pytest/test_qonnx.py::test_jet_tagging[Vivado]
  /home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/modelwrapper.py:93: UserWarning: Some old-style domain attributes were automatically converted to new-style,
                  i.e. domain=finn to domain=qonnx.custom_op.<general|fpgadataflow|...>
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED test_qonnx.py::test_cnv_2w2a[Quartus] - OSError: /home/hls4ml-user/hls4ml/test/pytest/hls4mlprj_qonnx_cnv-2w2a_Quar...
=========== 1 failed, 8 passed, 2106 warnings in 3564.98s (0:59:24) ============

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants