[Bug]: Older TopoStats U-Net Models Throw Error on Import

[MaxGamill-Sheffield]

Checklist

• Re-run analysis with topostats process --core 1.
• Describe the bug.
• Include the configuration file.
• Copy of the output.
• The exact command that failed. This is what you typed at the command line, including any options.
• TopoStats version, this is reported by topostats --version
• Operating System and Python Version

Describe the bug

.keras models that have been saved in older versions of keras struggle to load in newer versions, needing the compile=False flag, and any non-serialised objects, i.e. the iou_loss.

Copy of the output

layer issue

Serialisation issue:

[Tue, 10 Dec 2024 12:33:02] [ERROR ] [topostats] [processing.py] [185] [20241210_sup_nick_PJIP37_mixedpop_unknot_3ng_each_mgni.0_00013] : An error occurred during grain finding, skipping following steps.
Traceback (most recent call last):
File "/Users/laura/TopoStats/topostats/processing.py", line 177, in run_grains
grains.find_grains()
File "/Users/laura/TopoStats/topostats/grains.py", line 543, in find_grains
unet_mask, unet_labelled_regions = Grains.improve_grain_segmentation_unet(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/laura/TopoStats/topostats/grains.py", line 613, in improve_grain_segmentation_unet
unet_model = keras.models.load_model(unet_config["model_path"], custom_objects={"mean_iou": mean_iou})
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/laura/miniconda3/envs/topoly/lib/python3.11/site-packages/keras/src/saving/saving_api.py", line 196, in load_model
return legacy_h5_format.load_model_from_hdf5(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/laura/miniconda3/envs/topoly/lib/python3.11/site-packages/keras/src/legacy/saving/legacy_h5_format.py", line 155, in load_model_from_hdf5
**saving_utils.compile_args_from_training_config(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/laura/miniconda3/envs/topoly/lib/python3.11/site-packages/keras/src/legacy/saving/saving_utils.py", line 133, in compile_args_from_training_config
optimizer = optimizers.deserialize(optimizer_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/laura/miniconda3/envs/topoly/lib/python3.11/site-packages/keras/src/optimizers/init.py", line 65, in deserialize
return serialization_lib.deserialize_keras_object(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/laura/miniconda3/envs/topoly/lib/python3.11/site-packages/keras/src/saving/serialization_lib.py", line 581, in deserialize_keras_object
return deserialize_keras_object(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/laura/miniconda3/envs/topoly/lib/python3.11/site-packages/keras/src/saving/serialization_lib.py", line 718, in deserialize_keras_object
instance = cls.from_config(inner_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/laura/miniconda3/envs/topoly/lib/python3.11/site-packages/keras/src/optimizers/base_optimizer.py", line 955, in from_config
return cls(**config)
^^^^^^^^^^^^^
File "/Users/laura/miniconda3/envs/topoly/lib/python3.11/site-packages/keras/src/optimizers/adam.py", line 62, in init
super().init(
File "/Users/laura/miniconda3/envs/topoly/lib/python3.11/site-packages/keras/src/backend/tensorflow/optimizer.py", line 23, in init
super().init(*args, **kwargs)
File "/Users/laura/miniconda3/envs/topoly/lib/python3.11/site-packages/keras/src/optimizers/base_optimizer.py", line 86, in init
warnings.warn(
UserWarning: Argument decay is no longer supported and will be ignored.

Include the configuration file

diff:

unet_config:
    model_path: /Users/Maxgamill/Desktop/Uni/PhD/topo_cats/TopoStats/topostats/catsnet_cropped_2023-11-10_17-45-07.h5

To Reproduce

Run in new env, which should have a newer tensorflow installed

TopoStats Version

Git main branch

Python Version

3.1

Operating System

MacOS M1/M2 (post-2021)

Python Packages

No response

[ns-rse]

Wasn't something like DVC going to be used to manage the complexity of different models? I know I mentioned it to @SylviaWhittle at least in passing.

Things are only going to get more complicated as time progresses so might be worth starting off on a good path on this front.

P.S. Very old version of Python you're using 😜

Python Version

3.1