Issue with uint16 Images in Latest Nimbus-Inference Version

[RouvenHoefflin]

Report

Dear Nimbus Team,

Thank you again for developing and maintaining such an amazing tool!

After updating to the most recent version of Nimbus-Inference, I encountered an issue while running it on my uint16 images. Specifically, when executing Step 5 of the 1_Nimbus_Predict.ipynb notebook, I get the following error:

cell_table = nimbus.predict_fovs()

---------------------------------------------------------------------------
UFuncTypeError                            Traceback (most recent call last)
Cell In[11], line 1
----> 1 cell_table = nimbus.predict_fovs()

File [~/Nimbus-Inference/src/nimbus_inference/nimbus.py:230](http://hgn52.wexac.weizmann.ac.il:8888/home/labs/Nimbus-Inference/src/nimbus_inference/nimbus.py#line=229), in Nimbus.predict_fovs(self)
    228 print("Predictions will be saved in {}".format(self.output_dir))
    229 print("Iterating through fovs will take a while...")
--> 230 self.cell_table = predict_fovs(
    231     nimbus=self, dataset=self.dataset, output_dir=self.output_dir,
    232     save_predictions=self.save_predictions, batch_size=self.batch_size,
    233     test_time_augmentation=self.test_time_aug,
    234 )
    235 self.cell_table.to_csv(os.path.join(self.output_dir, "nimbus_cell_table.csv"), index=False)
    236 return self.cell_table

File [~/Nimbus-Inference/src/nimbus_inference/utils.py:494](http://hgn52.wexac.weizmann.ac.il:8888/home/labs/Nimbus-Inference/src/nimbus_inference/utils.py#line=493), in predict_fovs(nimbus, dataset, output_dir, suffix, save_predictions, batch_size, test_time_augmentation)
    492 instance_mask = dataset.get_segmentation(fov)
    493 for channel_name in tqdm(dataset.channels):
--> 494     mplex_img = dataset.get_channel_normalized(fov, channel_name)
    495     input_data = prepare_input_data(mplex_img, instance_mask)
    496     if dataset.magnification != nimbus.model_magnification:

File [~/Nimbus-Inference/src/nimbus_inference/utils.py:305](http://hgn52.wexac.weizmann.ac.il:8888/home/labs/Nimbus-Inference/src/nimbus_inference/utils.py#line=304), in MultiplexDataset.get_channel_normalized(self, fov, channel)
    303 else:
    304     norm_factor = np.quantile(mplex_img, 0.999)
--> 305 mplex_img /= norm_factor
    306 mplex_img = mplex_img.clip(0, 1)
    307 return mplex_img

UFuncTypeError: Cannot cast ufunc 'divide' output from dtype('float64') to dtype('<u2') with casting rule 'same_kind'

From my understanding, this error occurs because the tool is no longer compatible with uint16 images. Previously, I had no issues running the same workflow with these image formats.

Your example dataset with float32 images works without any errors.

Additionally, I noticed that the link to the changelog (https://nimbus-inference.readthedocs.io/latest/changelog.html) appears to be broken.

Could you please confirm whether the latest update no longer supports uint16 images? If so, is there a recommended solution or workaround to ensure compatibility with uint16 data?

Thank you in advance for your support!

All the best,
Rouven

Version information

---

alpineer NA
nimbus_inference 0.0.2
session_info 1.0.0

PIL 11.0.0
aiohappyeyeballs 2.4.3
aiohttp 3.11.8
aiosignal 1.3.1
anyio NA
asciitree NA
asttokens NA
async_timeout 5.0.1
attr 24.2.0
attrs 24.2.0
babel 2.11.0
backports NA
brotli 1.0.9
certifi 2024.08.30
charset_normalizer 3.3.2
comm 0.2.1
cv2 4.10.0
cycler 0.12.1
cython_runtime NA
datasets 3.1.0
dateutil 2.9.0.post0
debugpy 1.6.7
decorator 5.1.1
defusedxml 0.7.1
dill 0.3.8
exceptiongroup 1.2.0
executing 0.8.3
fastjsonschema NA
filelock 3.16.1
frozenlist 1.5.0
fsspec 2024.9.0
huggingface_hub 0.26.3
idna 3.7
imagecodecs 2024.9.22
imageio 2.36.1
ipykernel 6.29.5
ipywidgets 8.1.5
jaraco NA
jedi 0.19.1
jinja2 3.1.4
joblib 1.4.2
json5 0.9.25
jsonschema 4.23.0
jsonschema_specifications NA
jupyter_events 0.10.0
jupyter_server 2.14.1
jupyterlab_server 2.27.3
kiwisolver 1.4.7
lazy_loader 0.4
lmdb 1.5.1
lxml 5.3.0
markupsafe 2.1.3
matplotlib 3.9.2
more_itertools 10.3.0
mpl_toolkits NA
multidict 6.1.0
multiprocess 0.70.16
natsort 8.4.0
nbformat 5.10.4
numcodecs 0.13.1
numpy 1.26.4
overrides NA
packaging 24.1
pandas 2.2.3
parso 0.8.3
pkg_resources NA
platformdirs 3.10.0
prometheus_client NA
prompt_toolkit 3.0.43
propcache 0.2.0
psutil 5.9.0
pure_eval 0.2.2
pyarrow 18.1.0
pydev_ipython NA
pydevconsole NA
pydevd 2.9.5
pydevd_file_utils NA
pydevd_plugins NA
pydevd_tracing NA
pygments 2.15.1
pyometiff 1.0.1
pyparsing 3.2.0
pythonjsonlogger NA
pytz 2024.1
referencing NA
requests 2.32.3
rfc3339_validator 0.1.4
rfc3986_validator 0.1.1
rpds NA
scipy 1.14.1
send2trash NA
six 1.16.0
skimage 0.24.0
sniffio 1.3.0
socks 1.7.1
stack_data 0.2.0
tifffile 2024.9.20
torch 2.5.1+cu124
torchgen NA
tornado 6.4.1
tqdm 4.67.1
traitlets 5.14.3
typing_extensions NA
urllib3 2.2.3
wcwidth 0.2.5
websocket 1.8.0
xxhash NA
yaml 6.0.2
yarl 1.18.0
zarr 2.18.3
zmq 25.1.2
zoneinfo NA

IPython 8.27.0
jupyter_client 8.6.0
jupyter_core 5.7.2
jupyterlab 4.2.5

Python 3.10.15 | packaged by conda-forge | (main, Oct 16 2024, 01:24:24) [GCC 13.3.0]
Linux-5.14.0-162.6.1.el9_1.x86_64-x86_64-with-glibc2.34

Session information updated at 2024-12-03 00:16

[JLrumberger]

Hi @RouvenHoefflin,

Thanks for raising this issue. I'll fix it as soon as possible (hopefully by the end of this week).

Best,
Lorenz

[JLrumberger]

Should be fixed now. Could you give it a try?

[RouvenHoefflin]

Hi Lorenz,

thanks for the fast reply and bugfix. I updated Nimbus, however running 1_Nimbus_Predict.ipynb notebook on your example dataset throughs now an unrelated error:

# Prepare segmentation naming convention that maps a fov_path to the according segmentation label map
segmentation_naming_convention = prep_naming_convention(deepcell_output_dir)

# test segmentation_naming_convention
if os.path.exists(segmentation_naming_convention(fov_paths[0])):
    print("Segmentation data exists for fov 0 and naming convention is correct")
else:
    print("Segmentation data does not exist for fov 0 or naming convention is incorrect")

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[6], line 5
      2 segmentation_naming_convention = prep_naming_convention(deepcell_output_dir)
      4 # test segmentation_naming_convention
----> 5 if os.path.exists(segmentation_naming_convention(fov_paths[0])):
      6     print("Segmentation data exists for fov 0 and naming convention is correct")
      7 else:

File [~/Nimbus-Inference/src/nimbus_inference/nimbus.py:49](http://hgn06.wexac.weizmann.ac.il:8888/home/labs/Nimbus-Inference/src/nimbus_inference/nimbus.py#line=48), in prep_naming_convention.<locals>.segmentation_naming_convention(fov_path)
     47     raise ValueError(f"No segmentation data found for fov {fov_name}")
     48 if len(fnames) > 1:
---> 49     raise ValueError(f"Multiple segmentation data found for fov {fov_name}")
     50 return fnames[0]

ValueError: Multiple segmentation data found for fov fov0

I get the same error when trying to run it on my data. In both cases there are only unique segementation data for the fovs.
So I couldn't test the bugfix for the uint16 images.

Thanks for your help

Rouven

[JLrumberger]

Hi @RouvenHoefflin,

I need another one or two days to debug the errors that I introduced in the latest version. I didn't bump the pip version yet, so a quick fix for you to get back to the old behavior is to run pip install nimbus-inference and just use the pip version instead of the main branch of this repo. Sorry for the inconvenience and thanks so much for the feedback.

Best,
Lorenz

[RouvenHoefflin]

Hi Lorenz,

thanks so much for the update and the effort.
Do you have a rough timeline for the fix?
Thanks a lot!
All the best
Rouven

[JLrumberger]

Hi @RouvenHoefflin,

I think I fixed the bug, please have a try. Sorry for the delay.

Best,
Lorenz

[RouvenHoefflin]

Hi @JLrumberger,

Everything is fixed and works well now.

Thanks a lot. Really appreciate your help.

Rouven