itk.elastix_registration_method sometimes crashes silently

[tbirdso]

Overview

Registration with itk.ElastixRegistrationMethod or itk.elastix_registration_method sometimes crashes on Windows.

I am having difficulty pinning down the exact cause of the issue. It appears most frequently, but not exclusively, when the image is initialized from a NumPy array. I've been able to sometimes reproduce the issue across two Windows machines, across itk==v5.3.0 and itk==v5.4rc1, and itk-elastix versions 0.16.0, 0.18.0, and 0.19.0. It is unclear whether a resolution lies in ITK, Elastix, or ITKElastix.

Steps to Reproduce

Replace the image source in the first ITKElastix example with a NumPy-converted image.

fixed_image = itk.image_from_array(np.random.rand(10,10,10).astype(np.float32))
moving_image = itk.image_from_array(np.random.rand(10,10,10).astype(np.float32))

registered, parameters = itk.elastix_registration_method(fixed_image, moving_image )

Or, download test images and read in with itk.imread: https://data.kitware.com/#user/602bd4e62fa25629b97ba6ec/folder/6525e04e43427b2e1b65cba8

Expected Behavior

Registration completes without error.

Observed Behavior

The running Python process crashes. Logging to console or file shows output similar to below:

```py
>>> registered, parameters = itk.elastix_registration_method(s,t, log_to_console=True)
WARNING: The parameter "MovingInternalImagePixelType", requested at entry number 0, does not exist at all.
  The default value "float" is used instead.
ELASTIX version: 5.1.0
Command line options from ElastixBase:
-priority unspecified, so NORMAL process priority
-threads  unspecified, so all available threads are used
WARNING: The parameter "UseDirectionCosines", requested at entry number 0, does not exist at all.
  The default value "true" is used instead.

WARNING: The option "UseDirectionCosines" was not found in your parameter file.
  From elastix 4.8 it defaults to true!
This may change the behavior of your registrations considerably.

Command line options from TransformBase:
-t0       unspecified, so no initial transform used

Reading images...
Reading images took 0 ms.

WARNING: the fixed pyramid schedule is not fully specified!
  A default pyramid schedule is used.
WARNING: the moving pyramid schedule is not fully specified!
  A default pyramid schedule is used.
WARNING: The parameter "AutomaticTransformInitializationMethod", requested at entry number 0, does not exist at all.
  The default value "GeometricalCenter" is used instead.
Transform parameters are initialized as: [0, 0, 0]
Initialization of all components (before registration) took: 0 ms.
Preparation of the image pyramids took: 33 ms.

Resolution: 0
WARNING: The parameter "ShowExactMetricValue", requested at entry number 0, does not exist at all.
  The default value "false" is used instead.
... # other standard warnings ...
WARNING: The parameter "SigmoidScaleFactor", requested at entry number 0, does not exist at all.
  The default value "0.1" is used instead.
Elastix initialization of all components (for this resolution) took: 1 ms.
Initialization of AdvancedMattesMutualInformation metric took: 8 ms.
Starting automatic parameter estimation for AdaptiveStochasticGradientDescent ...
WARNING: The parameter "ASGDParameterEstimationMethod", requested at entry number 0, does not exist at all.
  The default value "Original" is used instead.
  Computing JacobianTerms ...
  Computing the Jacobian terms took 0.000548s
  NumberOfGradientMeasurements to estimate sigma_i: 11
  Sampling gradients ...
# silent crash

Sometimes when running within a dask.delayed function a Windows access violation error is printed to the console before exit:

Windows fatal exception: access violation

The behavior is reproducible in a standard Python console.

Platforms

Windows 10 and Windows 11

Versions

Python 3.8 and Python 3.10
itk v5.3.0, v5.4rc1
itk-elastix 0.16.0, 0.18.0, 0.19.0

Additional Notes

Perhaps there is an issue with the ITK NumPy bridge rather than in ITKElastix? I could not find any obvious differences between an image read from disk and an image obtained from a NumPy array. I am able to directly access pixels in the fixed and moving images without error.

[tbirdso]

cc @thewtex @N-Dekker

[dzenanz]

import itk
import numpy as np

fixed_image = itk.imread("target_image.mha")
moving_image = itk.imread("source_image.mha")

registered, parameters = itk.elastix_registration_method(fixed_image, moving_image, log_to_console=True)

crashes silently for me:

Elastix initialization of all components (for this resolution) took: 2 ms.
Initialization of AdvancedMattesMutualInformation metric took: 9 ms.
Starting automatic parameter estimation for AdaptiveStochasticGradientDescent ...
WARNING: The parameter "ASGDParameterEstimationMethod", requested at entry number 0, does not exist at all.
  The default value "Original" is used instead.
  Computing JacobianTerms ...
  Computing the Jacobian terms took 0.000753s
  NumberOfGradientMeasurements to estimate sigma_i: 11
  Sampling gradients ...

The random-generated images

import itk
import numpy as np

fixed_image = itk.image_from_array(np.random.rand(10,10,10).astype(np.float32))
moving_image = itk.image_from_array(np.random.rand(10,10,10).astype(np.float32))

registered, parameters = itk.elastix_registration_method(fixed_image, moving_image, log_to_console=True)

probably diverged:

...
53      -0.538808       0.000000        5.826118        0.081180        0.9
Time spent in resolution 0 (ITK initialization and iterating): 0.0855
Stopping condition: Error in metric.
Settings of AdaptiveStochasticGradientDescent in resolution 0:
( SP_a 122.348478 )
( SP_A 20.000000 )
( SP_alpha 1.000000 )
( SigmoidMax 1.000000 )
( SigmoidMin -0.307787 )
( SigmoidScale 0.000124 )


itk::ExceptionObject (0000002B603D5050)
Location: "ElastixTemplate - Run()"
File: D:\a\im\_skbuild\win-amd64-3.9\cmake-build\_deps\elx-src\Common\CostFunctions\itkAdvancedImageToImageMetric.hxx
Line: 901
Description: ITK ERROR: AdvancedMattesMutualInformationMetric(00000168BBE34690): Too many samples map outside moving image buffer: 406 / 2048


Error occurred during actual registration.


Traceback (most recent call last):
  File "m:\Elemance\mri-segmentation\testCrash.py", line 7, in <module>
    registered, parameters = itk.elastix_registration_method(fixed_image, moving_image, log_to_console=True)
  File "M:\Elemance\mri-segmentation\.venv\lib\site-packages\itk\support\helpers.py", line 176, in image_filter_wrapper
    return image_filter(*args, **kwargs)
  File "M:\Elemance\mri-segmentation\.venv\lib\site-packages\itk\itkElastixRegistrationMethodPython.py", line 1809, in elastix_registration_method
    return instance.__internal_call__()
  File "M:\Elemance\mri-segmentation\.venv\lib\site-packages\itk\ITKCommonBasePython.py", line 1437, in __internal_call__
    self.UpdateLargestPossibleRegion()
RuntimeError: D:\a\im\_skbuild\win-amd64-3.9\cmake-build\_deps\elx-src\Core\Main\itkElastixRegistrationMethod.hxx:380:
ITK ERROR: ElastixRegistrationMethod(00000168B5916D90): Internal elastix error: See elastix log (use LogToConsoleOn() or LogToFileOn()).