CMRR SBRef complex/multi-echo series sorting is broken

[bpinsard]

Summary

Warning: super weird edge case.

When acquiring mag+phase data with CMRR bold MB sequence, the SBRef get exported as a single series with both mag+phase dicoms, same goes if using multi-echo and/or combining the 2 ( eg. mag+phase * 3 echoes = 6 dicom files)
However, MosaicWrapper.is_same_series will take into account the dicoms ImageType in signature, and thus each file is considered a separate series at that point:

heudiconv/heudiconv/dicoms.py

Line 230 in cb2fd91

if mw.is_same_series(mwgroup[idx]):

because ImageType will have M/P , TE1/TE2/.../TEn, even if they still generate a single Seqinfo with heudiconv default grouping here:

heudiconv/heudiconv/dicoms.py

Lines 115 to 117 in cb2fd91

	series_id = (
	int(mw.dcm_data.SeriesNumber), mw.dcm_data.ProtocolName
	)

So there is a discrepancy between the grouping in heudiconv and nibabel, not sure if and where it should be fixed.

This results in the last dicom file of the series being used to create the seqinfo of that series (due to the dict creation below with repeated keys), leading to inconsistent Seqinfos.

heudiconv/heudiconv/dicoms.py

Line 250 in cb2fd91

group_map = dict(zip(groups[0], groups[1]))

This issue is related to #519 and #535
In such cases, the Seqinfo values will depend on the order of the provided files, and that for instance when data are stored based on SOPInstanceUID, Siemens ones are of variable length and not reliably related to the acquisition/reconstruction timing when sorting alphabetically (so one cannot assume mag/phase or TE orders). Example:

MR.1.3.12.2.1107.5.2.43.123456.2022032116191110159432266
MR.1.3.12.2.1107.5.2.43.123456.202203211619119969932265

I think the results of heudiconv should be independent of the order of the files provided.
Maybe the files within a series should first be sorted with ContentTime, so the order is consistent with the acquisition and reconstruction order?
The other option would be that I code a custom grouping that will match nibabel one (splitting series based on ImageType), but it will require to rewrite all the logic for complex, multi-echo, ... that is already implemented in heudiconv.

Let me know if that is clear or not. I know this is very convoluted.
Even if there is no obvious solution, I post it as a reference for anyone having the same problem and not being stubborn enough to read and trace what happens in the code.

Platform details:

Choose one:

• Local environment

• Container

• Heudiconv version:

0.10.0

[DVSneuro]

I'm getting the same issue with our multi-echo data. We're also collecting magnitude and phase data in the hopes that we can easily use the phase data in the future (https://neurostars.org/t/processing-for-complex-valued-fmri-data/5093; thanks to @tsalo for the helpful information on NeuroStars). The functionals appear to be stored as separate mag and phase images, and we can grab the magnitude with the protocol name in the heuristic.py. But I think the SBref image comes off the scanner as a complex image and HeuDiConv separates that into part-phase and part-mag images.

Curiously, fMRIprep didn't have a problem with our oddly-named SBref images (sub-10035_task-sharedreward_acq-mb6me4_echo-1_part-phase_sbref and sub-10035_task-sharedreward_acq-mb6me4_echo-1_part-mag_sbref), but I think it averaged the phase and magnitude images together before motion correction. As a temporary workaround, I just deleted the part-phase_sbref images and renamed the part-mag_sbref images.

I'm using HeuDiConv version 0.13.1.

[yarikoptic]

But I think the SBref image comes off the scanner as a complex image and HeuDiConv separates that into part-phase and part-mag images.

well, it then would be dcm2niix which has such power since heudiconv doesn't really do the conversion/splitting. Might be worth checking with @rordenlab