DICOM import failed with datalad hirni-import-dcm

[lsempf]

Hi guys,
We have successfully imported most of our acquisitions with datalad hirni-import-dcm. However, two acquisitions resulted in errors. The datalad hirni commands do not differ and with the DICOM acquisitions everything seems to be OK.

datalad hirni-import-dcm --anon-subject 001 /path/to/file.tar ab01
[Hirni ADDONS] Running following command:
[Hirni ADDONS] datalad hirni-import-dcm --anon-subject 001 /path/to/file.tar ab01
[INFO ] Creating a new annex repo at /sourcedata/ab01/dicoms
[INFO ] Adding content of the archive ab01_dicomsorted_nr111.tar into annex <AnnexRepo path=/sourcedata/ab01/dicoms (<class 'datalad.support.annexrepo.A$
[INFO ] Finished adding ab01_dicomsorted_nr111.tar: Files processed: 3200, +annex: 3200
Metadata aggregation: 0%| | 0.00/1.00 [00:00<?, ? datasets/s
[ERROR ] Failed to get metadata (dicom): Unknown Value Representation '0x00 0x00' in tag (0000, 0000) [dataelem.py:DataElement_from_raw:759] [meta_extract(/sourcedata/ab01/dicoms)] ########################################################8| 3.20k/3.20k [03:28<00:00, 25.1 Files/s]

datalad hirni-import-dcm --anon-subject 002 /path/to/file.tar yx99
[Hirni ADDONS] Running following command:
[Hirni ADDONS] datalad hirni-import-dcm --anon-subject 002 /path/to/file.tar yx99
[INFO ] Creating a new annex repo at /sourcedata/yx99/dicoms
[INFO ] Adding content of the archive yx99_dicomsorted_nr222.tar into annex <AnnexRepo path=/sourcedata/yx99/dicoms (<class 'datalad.support.annexrepo.A$
[INFO ] Finished adding yx99_dicomsorted_nr222.tar: Files processed: 3331, +annex: 3331
here
[ERROR ] 'id' [spec_helpers.py:get_specval:33] (KeyError)

The DICOMS are imported - with all subfolders of the different measurements. But in both cases the studyspec.json files are not created.

I would be grateful for any help.

Best,
Linda

[bpoldrack]

Thanks for reporting, @lsempf!

At a first glance I'm not quite sure about the first error ( Unknown Value Representation ), but it seems to me that there's something wrong in the DICOM headers of that acquisition (or a part of it). Need to look into what the (0000, 0000) tag in DICOM is for - haven't seen this before. However, that smells like an actually corrupted DICOM file. Can you read all the DICOMS (or just their headers) in that acquisition with any software? And if so: What's the value for (0000, 0000)?
If we can figure out what's wrong in this case, that would be greatly appreciated!

The second error (KeyError) indicates that the DICOMS are missing a header field we expected to be there (Likely SeriesNumber). That's an issue with datalad-hirni indeed - we need to figure out how exactly to deal with missing ID fields of all kinds (which - for some fields of that kind - is likely to be caused by some anonymization procedure manipulating the DICOMS).
In this second case you could currently only manually build the specification file. Whether that's doable for you depends of course on how much you know about the acquisition and those specification files. I can help only with the second part. I'm working on a solution within hirni, but I can't promise how quickly I'll get that done.

[bpoldrack]

Note to myself: Considering this a bug since we really need to be more robust against faulty or unexpected headers.