MD Express dicom seg is flipped when displayed on OHIF viewer

[AHarouni]

I have added OHIF viewer 2.x used by monai label. I made OHIF use orthanc instance used by MD express.
Things worked great, but unfortunately the returned dicom seg appears to be flipped as shown below

I tired with some other TCIA data but it seems to be always flipped or sometimes OHIF can't even view the dicom seg giving the error

DICOM Segmentation Loader
Source Image Sequence information missing: individual SEG frames are out of plane. This is not yet supported. Aborting segmentation loading.

It seems like the dicom seg is some how out of the ROI or plane.
@MMelQin pointed out it might be related to this issue
Tagging @dbericat

[MMelQin]

I have added OHIF viewer 2.x used by monai label. I made OHIF use orthanc instance used by MD express. Things worked great, but unfortunately the returned dicom seg appears to be flipped as shown below

I tired with some other TCIA data but it seems to be always flipped or sometimes OHIF can't even view the dicom seg giving the error

DICOM Segmentation Loader
Source Image Sequence information missing: individual SEG frames are out of plane. This is not yet supported. Aborting segmentation loading.

It seems like the dicom seg is some how out of the ROI or plane. @MMelQin pointed out it might be related to this issue Tagging @dbericat

That is the liver dataset converted into DICOM from the Medical Decathlon training and validation dataset, 014, in NIfTI. Whoever did not original conversion to NIfTI, re-orientated the original DICOM image, but then set the orientation as if not done. It gave me headache as the image does not follow either Rad convention (LPS) nor Nuero convention (RAS).

Please find a CT abdomen series from TCIA or the Rad Image Net, and we can deep dive on this one. Even with the specific 014 converted DICOM series, the seg and the original series line up in 3D slicer.

BTW, for the example application in the App SDK, I have changed to use TCIA dataset, where I can.

[AHarouni]

Hi @MMelQin
"Please find a CT abdomen series from TCIA or the Rad Image Net"
I already tried 2 abdomen CTs from TCIA and got error

DICOM Segmentation Loader
Source Image Sequence information missing: individual SEG frames are out of plane. This is not yet supported. Aborting segmentation loading.

I would ask to at least change the sample data so it can work correctly.

[MMelQin]

Hi @MMelQin "Please find a CT abdomen series from TCIA or the Rad Image Net" I already tried 2 abdomen CTs from TCIA and got error

DICOM Segmentation Loader
Source Image Sequence information missing: individual SEG frames are out of plane. This is not yet supported. Aborting segmentation loading.

I would ask to at least change the sample data so it can work correctly.

Thanks. I double checked the Seg over the original, with 3D Slicer, and the images look good even with the sample data set. MicroDicom shows the liver on the correct side (Rad convention), but it does not support overlay.

In your screenshot, the position and shape of the displayed seg looks good, too, but the scale is off as compared to the original, for unknow reason (the model itself was grabbed from NGC, which was trained with old monai version, and the extra piece of "liver" unexpectedly showed up)

[MMelQin]

Also, looks to me that in your screenshot, the seg slice is NOT overlaid on the correct corresponding original instance. The shape of the abdomen looks different than the one shown on 3D slicer.

[AHarouni]

Hi @MMelQin

I think slicer is a smarter viewer than OHIF. I am not sure why things are not displaying correctly in OHIF. However, if I trigger inference from monai label using ohif it generates dicom seg and OHIF displays it correctly.

So may be the difference between monai label and monai deploy is in the libraries they are using ? Last I checked monai label is using itkimage2segimage https://github.com/Project-MONAI/MONAILabel/blob/main/monailabel/datastore/utils/convert.py#L174

I think deploy switched to highdicom, does it use the same util in the backend? if not may be that is the root cause of discrepancy.

[MMelQin]

Hi @MMelQin

I think slicer is a smarter viewer than OHIF. I am not sure why things are not displaying correctly in OHIF. However, if I trigger inference from monai label using ohif it generates dicom seg and OHIF displays it correctly.

So may be the difference between monai label and monai deploy is in the libraries they are using ? Last I checked monai label is using itkimage2segimage https://github.com/Project-MONAI/MONAILabel/blob/main/monailabel/datastore/utils/convert.py#L174

I think deploy switched to highdicom, does it use the same util in the backend? if not may be that is the root cause of discrepancy.

@AHarouni No, highdicom does not use ITK, and directly uses pydicom. The highdicom OSS contributors include the who's who in DICOM, including David Clunie, so I'd have high confidence in it. Besides, outputs from the package have been validated by the DICOM validation tool (David's Dicom 3 tool).

Recall that the initial App SDK (also Clara Deploy) DICOM Seg Writer, using pydicom only, generated instances that can be properly displayed in all compatible viewer, including OHIF, though that version of the writer had some minor issues with type 3 tags and multi-seg. So there must be something additional in the highdicom output that OHIF fails to properly parse, as evidenced by the issue on its board. @CPBridge may know more on this issue.

[CPBridge]

Just seeing this now. Thanks for digging into this @MMelQin and @AHarouni

There is a huge amount of flexibility in the DICOM Seg standard and some viewers do not appear to be correctly supporting all of the things that can happen. Spatially, the only real requirement is that the source image(s) and the segmentation image are defined within the same frame of reference. Beyond that, the seg image may be stored at a different resolution, flipped in any direction, or even have undergone arbitrary 3D rotations etc just as long as the Image Position Patient and Image Orientation Patient information in the Per Frame Functional Groups (or Shared Functional Groups) correctly describes that transformation. I suspect OHIF is making some incorrect assumptions that the orientation of the segmentation image always matches that of the source images (which it often will unless some flipping or other transformation has occurred during preprocessing). Generally speaking, Slicer handles all this stuff well and so I'm more inclined to trust it. Therefore I expect this is related to OHIF/Viewers#2833 as others have suggested and would encourage you to comment there too.

However at this point this is speculation on my part. If you are able to send me the relevant DICOM files for the segmentation and the source images, I will try to take a look at see if this reveals anything obvious. If you do not have my email it's chrisbridge44 at gmail.