Trying to make a C-MOVE Q/R to download multiple studies by giving a list of Accession Numbers

[deniu5]

Hi,

I've been trying for several weeks to download medical studies by providing the AN. My code works fine for downloading a single study, but I need to download multiple studies. Using the same code, I implemented a simple loop where each element is an Accession Number. The code works and downloads the studies, but the issue is that during the download, it starts several download processes simultaneously, causing some DICOM files to be saved in the wrong study folders.

My code works fine for one study, but when I try to download multiple studies by passing only the Accession Numbers in a list and looping through them, the DICOM files from each study end up getting mixed. This happens because the download of one study starts before the previous one finishes. I tried starting an SCP server (scp = ae.start_server(('', 104), block=True, evt_handlers=handlers)), but it freezes after downloading the first study.

Here are some the functions that I'm using:

handle_store function

def generador_handle_store(accession_number):
stored_count = 0 # Contador local de imágenes almacenadas por estudio

def handle_store(event):
    nonlocal stored_count  # Usamos nonlocal para que el contador sea independiente por cada estudio

    # Crear la carpeta del estudio si no existe
    study_dir = os.path.join(output_dir, accession_number)
    os.makedirs(study_dir, exist_ok=True)

    # Obtener el dataset del DICOM
    ds = event.dataset
    ds.file_meta = event.file_meta

    # Guardar el fichero DICOM en la carpeta del estudio
    file_path = os.path.join(study_dir, f"{stored_count}.dcm")
    ds.save_as(file_path, write_like_original=False)
    print(f"Imagen guardada en: {file_path}")

    stored_count += 1  # Incrementar el contador

    return 0x0000  # Status success

return handle_store

and this download function

def descargar_estudios(pacs_ip, pacs_port_qr, pacs_ae_title, local_ae_title, lista_accession_numbers):
ae = AE(ae_title=local_ae_title)

for context in StoragePresentationContexts:
    ae.add_supported_context(context.abstract_syntax)

ae.add_requested_context(StudyRootQueryRetrieveInformationModelMove)
ae.add_requested_context(UltrasoundMultiFrameImageStorage)
ae.add_requested_context(Verification)

for accession_number in lista_accession_numbers:
    print(f"\nDescargando estudio con AN: {accession_number}")

    # Crear un manejador independiente para cada estudio
    handlers = [(evt.EVT_C_STORE, generador_handle_store(accession_number))]

    # Iniciar el SCP
    scp = ae.start_server(('', port), block=False, evt_handlers=handlers)

    # Asociar con el servidor PACS
    assoc = ae.associate(pacs_ip, pacs_port_qr, ae_title=pacs_ae_title)

    if assoc.is_established:
        print("Conexión exitosa con el servidor PACS.")
        print(f"Contextos soportados: {assoc.accepted_contexts}")
        

        # Crear dataset para la consulta C-MOVE
        ds = Dataset()
        ds.QueryRetrieveLevel = "STUDY"
        ds.AccessionNumber = accession_number
        

        # Enviar la solicitud C-MOVE
        responses = assoc.send_c_move(ds, move_aet=local_ae_title, query_model=StudyRootQueryRetrieveInformationModelMove, priority=0)  

        # Manejar las respuestas
        for status, identifier in responses:
            print(f"Estado recibido: {status}")
            if hasattr(status, 'Status'):
                if status.Status in (0xFF00, 0xFF01):
                    print("Estudio en proceso de descarga...")
                elif status.Status == 0x0000:
                    print("Descarga del estudio completa.")
                else:
                    print(f"Error durante la transferencia: {status.Status} (Código de error)")
            else:
                print("Error: El objeto status no tiene un atributo 'Status'")

    else:
        print("No se pudo establecer conexión con el servidor PACS.")
    
    assoc.release()

    # Asegurar que todas las imágenes del estudio actual se descarguen antes de pasar al siguiente
    scp.shutdown()

[medihack]

Why don't you store the datasets in you handle_store callback in different subfolders, depending on some values in the dataset like PatientID, StudyDate, and so on?

[deniu5]

In fact, that's what I'm doing. In the handle_store event, before saving the dataset, I'm creating a folder with the corresponding accession number. The result is what I want, but the problem is that the downloading is working asynchronously, and this sometimes causes some DICOM files to get mixed with others. I already tried setting block=True, but in that case, it doesn't download more than one study:

scp = ae.start_server(('', 104), block=True, evt_handlers=handlers)

[scaramallion]

The peer really shouldn't be using async unless asked to do so. pynetdicom doesn't support async, but you can try sending an asynchronous operations request to the peer during association (via the ext_neg arg in AE.associate()) telling it to not use async (invoked/performed both 1). I doubt it will work, but its all you can do.

[deniu5]

I'll try to investigate that issue. If it works, I'll share the 'solution'.

Thanks very much for your attention!!

[deniu5]

I've tried doing that, but it doesn't work. I've also tried using 'start_server' with the block option set to True, and while it downloads the first study, it continues downloading the others into the same folder without stopping, until all the studies are downloaded in the same directory. Something that I don't really understand is that during the download, the files are being overwritten.

Image sabe in: K:\ESTUDIOS DICOM\AN\0
Image sabe in: K:\ESTUDIOS DICOM\AN\1
Image sabe in: K:\ESTUDIOS DICOM\AN\2
Image sabe in: K:\ESTUDIOS DICOM\AN\3
Image sabe in: K:\ESTUDIOS DICOM\AN\3
Image sabe in: K:\ESTUDIOS DICOM\AN\5
Image sabe in: K:\ESTUDIOS DICOM\AN\6
Image sabe in: K:\ESTUDIOS DICOM\AN\6

the number is the name of DICOM file and it must be set the stored_count variable for each downloading process, but I don't why is not reaching the correct 'enumaration'.

I've read the handlers documentation, but I don't managed to know what it happening here to try to solve this issue.

Thanks in advance for your attention!!

[deniu5]

And there is another doubt that would be goodto know the reason why it doesn't work, and is that the conditionals of the previous code I'd passed it:

Manejar las respuestas

    for status, identifier in responses:
        print(f"Estado recibido: {status}")
        if hasattr(status, 'Status'):
            if status.Status in (0xFF00, 0xFF01):
                print("Estudio en proceso de descarga...")
            elif status.Status == 0x0000:
                print("Descarga del estudio completa.")
            else:
                print(f"Error durante la transferencia: {status.Status} (Código de error)")
        else:
            print("Error: El objeto status no tiene un atributo 'Status'")

else:
    print("No se pudo establecer conexión con el servidor PACS.")

is to managed the responses and to know how the Q/R query is going. Most of the time (rarely manage the 0000 response) never get any of those conditions. How can I solve/fix this?

Another problem has come up. I'm trying to filter the studies (UltrasoundMultiFrameImage), while dowloading, removing the static images and trying to leave the DICOM files that store multiframes, but I can't manage to do it correctly, using the NumberOfFrames tag (0028, 0008).

dicom_data = pydicom.dcmread(file_path)
if (0x0028, 0x0008) not in dicom_data:
print(f"Eliminado el fichero {os.path.basename(file_path)} dicom")
os.remove(file_path)

[scaramallion]

NumberOfFrames might be 1, you might want something like

ds = dcmread(path)
if ds.get("NumberOfFrames", 1) == 1:
    # Skip

[scaramallion]

You really should be getting 0xFF00 or 0xFF01 pending responses, can you attach the anonymised log output from debug_logger()?

[deniu5]

And also it might be with the FrameTime tag (0018,1063)? I noticed that the static images don't have this tag.

Here is the debug_logger() output:

INFO - Requesting Association
DEBUG - Request Parameters:
DEBUG - ======================= OUTGOING A-ASSOCIATE-RQ PDU ========================
DEBUG - Our Implementation Class UID: [ANONYMIZED_UID]
DEBUG - Our Implementation Version Name: PYNETDICOM_210
DEBUG - Application Context Name: 1.2.840.10008.3.1.1.1
DEBUG - Calling Application Name: [ANONYMIZED_AE_TITLE]
DEBUG - Called Application Name: [ANONYMIZED_AE_TITLE]
DEBUG - Our Max PDU Receive Size: 16382
DEBUG - Presentation Contexts:
DEBUG - Context ID: 1 (Proposed)
DEBUG - Abstract Syntax: =Study Root Query/Retrieve Information Model - MOVE
DEBUG - Proposed SCP/SCU Role: Default
DEBUG - Proposed Transfer Syntaxes:
DEBUG - =Implicit VR Little Endian
DEBUG - =Explicit VR Little Endian
DEBUG - =Deflated Explicit VR Little Endian
DEBUG - =Explicit VR Big Endian
DEBUG - Context ID: 3 (Proposed)
DEBUG - Abstract Syntax: =Ultrasound Multi-frame Image Storage
DEBUG - Proposed SCP/SCU Role: Default
DEBUG - Proposed Transfer Syntaxes:
DEBUG - =Implicit VR Little Endian
DEBUG - =Explicit VR Little Endian
DEBUG - =Deflated Explicit VR Little Endian
DEBUG - =Explicit VR Big Endian
DEBUG - Context ID: 5 (Proposed)
DEBUG - Abstract Syntax: =Verification SOP Class
DEBUG - Proposed SCP/SCU Role: Default
DEBUG - Proposed Transfer Syntaxes:
DEBUG - =Implicit VR Little Endian
DEBUG - =Explicit VR Little Endian
DEBUG - =Deflated Explicit VR Little Endian
DEBUG - =Explicit VR Big Endian
DEBUG - Requested Extended Negotiation: None
DEBUG - Requested Common Extended Negotiation: None
DEBUG - Requested Asynchronous Operations Window Negotiation: None
DEBUG - Requested User Identity Negotiation: None
DEBUG - ========================== END A-ASSOCIATE-RQ PDU ==========================
DEBUG - Accept Parameters:
DEBUG - ======================= INCOMING A-ASSOCIATE-AC PDU ========================
DEBUG - Their Implementation Class UID: [ANONYMIZED_UID]
DEBUG - Their Implementation Version Name: RAIMDicom_1.0
DEBUG - Application Context Name: 1.2.840.10008.3.1.1.1
DEBUG - Calling Application Name: [ANONYMIZED_AE_TITLE]
DEBUG - Called Application Name: [ANONYMIZED_AE_TITLE]
DEBUG - Their Max PDU Receive Size: 131072
DEBUG - Presentation Contexts:
DEBUG - Context ID: 1 (Accepted)
DEBUG - Abstract Syntax: =Study Root Query/Retrieve Information Model - MOVE
DEBUG - Accepted SCP/SCU Role: Default
DEBUG - Accepted Transfer Syntax: =Implicit VR Little Endian
DEBUG - Context ID: 3 (Accepted)
DEBUG - Abstract Syntax: =Ultrasound Multi-frame Image Storage
DEBUG - Accepted SCP/SCU Role: Default
DEBUG - Accepted Transfer Syntax: =Implicit VR Little Endian
DEBUG - Context ID: 5 (Accepted)
DEBUG - Abstract Syntax: =Verification SOP Class
DEBUG - Accepted SCP/SCU Role: Default
DEBUG - Accepted Transfer Syntax: =Implicit VR Little Endian
DEBUG - Accepted Extended Negotiation: None
DEBUG - Accepted Asynchronous Operations Window Negotiation: None
DEBUG - User Identity Negotiation Response: None
DEBUG - ========================== END A-ASSOCIATE-AC PDU ==========================
INFO - Association Accepted
INFO - Sending Move Request: MsgID 1
INFO -
INFO - # Request Identifier
INFO - (0008,0050) SH [ANONYMIZED_ACCESSION_NUMBER] # 1 AccessionNumber
INFO - (0008,0052) CS [STUDY] # 1 QueryRetrieveLevel
INFO -
DEBUG - ========================== OUTGOING DIMSE MESSAGE ==========================
DEBUG - Message Type : C-MOVE RQ
DEBUG - Message ID : 1
DEBUG - Affected SOP Class UID : Study Root Query/Retrieve Information Model - MOVE
DEBUG - Move Destination : [ANONYMIZED_AE_TITLE]
DEBUG - Identifier : Present
DEBUG - Priority : Medium
DEBUG - ============================ END DIMSE MESSAGE =============================
DEBUG - Request Parameters:
DEBUG - ======================= INCOMING A-ASSOCIATE-RQ PDU ========================
DEBUG - Their Implementation Class UID: [ANONYMIZED_UID]
DEBUG - Their Implementation Version Name: RAIMDicom_1.0
DEBUG - Application Context Name: 1.2.840.10008.3.1.1.1
DEBUG - Calling Application Name: [ANONYMIZED_AE_TITLE]
DEBUG - Called Application Name: [ANONYMIZED_AE_TITLE]
DEBUG - Their Max PDU Receive Size: 16384
DEBUG - Presentation Contexts:
DEBUG - Context ID: 1 (Proposed)
DEBUG - Abstract Syntax: =Ultrasound Multi-frame Image Storage
DEBUG - Proposed SCP/SCU Role: Default
DEBUG - Proposed Transfer Syntax:
DEBUG - =JPEG Baseline (Process 1)
DEBUG - Context ID: 3 (Proposed)
DEBUG - Abstract Syntax: =Ultrasound Multi-frame Image Storage
DEBUG - Proposed SCP/SCU Role: Default
DEBUG - Proposed Transfer Syntax:
DEBUG - =Implicit VR Little Endian
DEBUG - Requested Extended Negotiation: None
DEBUG - Requested Common Extended Negotiation: None
DEBUG - Requested Asynchronous Operations Window Negotiation: None
DEBUG - Requested User Identity Negotiation: None
DEBUG - ========================== END A-ASSOCIATE-RQ PDU ==========================
INFO - Accepting Association
DEBUG - Accept Parameters:
DEBUG - ======================= OUTGOING A-ASSOCIATE-AC PDU ========================
DEBUG - Our Implementation Class UID: [ANONYMIZED_UID]
DEBUG - Our Implementation Version Name: PYNETDICOM_210
DEBUG - Application Context Name: 1.2.840.10008.3.1.1.1
DEBUG - Responding Application Name: [ANONYMIZED_AE_TITLE]
DEBUG - Our Max PDU Receive Size: 16382
DEBUG - Presentation Contexts:
DEBUG - Context ID: 1 (Rejected - Transfer Syntax Not Supported)
DEBUG - Abstract Syntax: =Ultrasound Multi-frame Image Storage
DEBUG - Context ID: 3 (Accepted)
DEBUG - Abstract Syntax: =Ultrasound Multi-frame Image Storage
DEBUG - Accepted SCP/SCU Role: Default
DEBUG - Accepted Transfer Syntax: =Implicit VR Little Endian
DEBUG - Accepted Extended Negotiation: None
DEBUG - Accepted Asynchronous Operations Window Negotiation: None
DEBUG - User Identity Negotiation Response: None
DEBUG - ========================== END A-ASSOCIATE-AC PDU ==========================
DEBUG - pydicom.read_dataset() TransferSyntax="Little Endian Implicit"
INFO - Received Store Request
DEBUG - ========================== INCOMING DIMSE MESSAGE ==========================
DEBUG - Message Type : C-STORE RQ
DEBUG - Presentation Context ID : 3
DEBUG - Message ID : 1
DEBUG - Affected SOP Class UID : Ultrasound Multi-frame Image Storage
DEBUG - Affected SOP Instance UID : [ANONYMIZED_SOP_UID]
DEBUG - Data Set : Present
DEBUG - Priority : Low
DEBUG - ============================ END DIMSE MESSAGE =============================
DEBUG - pydicom.read_dataset() TransferSyntax="Little Endian Implicit"
DEBUG - pydicom.read_dataset() TransferSyntax="Little Endian Implicit"
INFO - Received Store Request
DEBUG - ========================== INCOMING DIMSE MESSAGE ==========================
DEBUG - Message Type : C-STORE RQ
DEBUG - Presentation Context ID : 3
DEBUG - Message ID : 2
DEBUG - Affected SOP Class UID : Ultrasound Multi-frame Image Storage
DEBUG - Affected SOP Instance UID : [ANONYMIZED_SOP_UID]
DEBUG - Data Set : Present
DEBUG - Priority : Low
DEBUG - ============================ END DIMSE MESSAGE =============================
DEBUG - pydicom.read_dataset() TransferSyntax="Little Endian Implicit"
I: Image save in : K:\ESTUDIOS DICOM\AN\23
INFO - Association Released

[scaramallion]

There should be at least a final C-MOVE success/warning/failure status. The pending statuses are optional, though.

[deniu5]

I agree with that, but the 'importance' to show a success or failure status is to register a log in case there is something wrong during the downloading (this should download a big quantity of studies, in the order of 200-300)