monailabel/monailabel_radiology_spleen_segmentation_OHIF.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copyright (c) MONAI Consortium  \n",
    "Licensed under the Apache License, Version 2.0 (the \"License\");  \n",
    "you may not use this file except in compliance with the License.  \n",
    "You may obtain a copy of the License at  \n",
    "&nbsp;&nbsp;&nbsp;&nbsp;http://www.apache.org/licenses/LICENSE-2.0  \n",
    "Unless required by applicable law or agreed to in writing, software  \n",
    "distributed under the License is distributed on an \"AS IS\" BASIS,  \n",
    "WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  \n",
    "See the License for the specific language governing permissions and  \n",
    "limitations under the License. \n",
    "\n",
    "# MONAI Label Radiology App - OHIF Spleen Segmentation Example\n",
    "\n",
    "***The OHIF + MONAI Label integration with spleen segmentation***\n",
    "\n",
    "In this notebook, we show end-to-end setup of the web-based OHIF viewer and MONAI Label plugin. The spleen segmentation is demonstrated.\n",
    "\n",
    "<p align = \"center\"><img src=\"./figures/monailabel_radiology_spleen_segmentation_OHIF/ohif0.png\" alt=\"drawing\" width=\"900\"/></p>\n",
    "\n",
    "## 1. Prepare MONAI Label"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup environment\n",
    "\n",
    "### Prerequisites\n",
    "- **Install MONAI Label** weekly preview release: \n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "!pip install monailabel"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1 Download Radiology app\n",
    "Choose \"radiology\" as the app"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "!monailabel apps --download --name radiology --output apps"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Setup Orthanc DICOM Server\n",
    "\n",
    "OHIF works well with the web-based Orthanc DICOM Server. Orthanc provides a simple, and powerful standalone DICOM server. It is designed to improve the DICOM flows in hospitals and to support research about the automated analysis of medical images. For more details on Orthanc, please refer to the [Orthanc](https://www.orthanc-server.com/index.php) page. \n",
    "\n",
    "The official [download page](https://www.orthanc-server.com/download.php) provides detailed steps for installing Orthanc.\n",
    "\n",
    "```bash\n",
    "# Install orthanc and dicomweb plugin\n",
    "sudo apt-get install orthanc orthanc-dicomweb -y\n",
    "\n",
    "# stop the existing Orthanc instance if there is one\n",
    "sudo service orthanc stop\n",
    "\n",
    "# setup and upgrade Orthanc libraries\n",
    "sudo wget https://orthanc.uclouvain.be/downloads/linux-standard-base/orthanc/1.12.3/Orthanc --output-document /usr/sbin/Orthanc\n",
    "sudo rm -f /usr/share/orthanc/plugins/*.so\n",
    "\n",
    "sudo wget https://orthanc.uclouvain.be/downloads/linux-standard-base/orthanc/1.12.3/libServeFolders.so --output-document /usr/share/orthanc/plugins/libServeFolders.so\n",
    "sudo wget https://orthanc.uclouvain.be/downloads/linux-standard-base/orthanc/1.12.3/libModalityWorklists.so --output-document /usr/share/orthanc/plugins/libModalityWorklists.so\n",
    "sudo wget https://orthanc.uclouvain.be/downloads/linux-standard-base/orthanc-dicomweb/1.16/libOrthancDicomWeb.so --output-document /usr/share/orthanc/plugins/libOrthancDicomWeb.so\n",
    "\n",
    "# start\n",
    "sudo service orthanc restart\n",
    "```\n",
    "\n",
    "After the above steps, the Orthanc will up by default at\n",
    "\n",
    "**http://127.0.0.1:8042**\n",
    "\n",
    "The index page of Orthanc DICOM Web page:\n",
    "\n",
    "<p align = \"center\"><img src=\"./figures/monailabel_radiology_spleen_segmentation_OHIF/ohif1.png\" alt=\"drawing\" width=\"900\"/></p>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Prepare Dicom Sample data\n",
    "\n",
    "### 3.1 Download Spleen CT Data\n",
    "Download MSD Spleen dataset as the sample dataset using monailabel API. The task is the volumetric (3D) segmentation of the spleen from CT image. The segmentation of the spleen is formulated as the voxel-wise 2-class classification. Each voxel is predicted as either foreground (spleen) or background. The dataset is from the 2018 MICCAI challenge."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "!monailabel datasets --download --name Task09_Spleen --output datasets"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 Convert to Dicom Format\n",
    "\n",
    "To use the DICOM server, the NIFTI format data needs to be converted to DICOM format, [PyPlastimatch](https://github.com/ImagingDataCommons/pyplastimatch) is a python wrapper for [Plastimatch](https://plastimatch.org/) which is an open tool to generate simple DICOM data from NIFTI file. The following commands show how to convert a sample DICOM data."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "# Install `pyplastimatch` NIFTI to DICOM converter\n",
    "!pip install pyplastimatch\n",
    "\n",
    "import pyplastimatch as pypla\n",
    "from pyplastimatch.utils.install import install_precompiled_binaries\n",
    "\n",
    "install_precompiled_binaries()\n",
    "\n",
    "# convert one of the NIFTI images to DICOM: name: <patient1>, output folder: <dicom_output>\n",
    "convert_args_ct = {\n",
    "    \"input\": \"datasets/Task09_Spleen/imagesTs/spleen_1.nii.gz\",\n",
    "    \"patient-id\": \"patient1\",\n",
    "    \"output-dicom\": \"dicom_output\",\n",
    "}\n",
    "pypla.convert(verbose=True, **convert_args_ct)"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3 Upload to Orthanc\n",
    "\n",
    "- Click `Upload` on the top right corner of Orthanc index page. \n",
    "\n",
    "<p align = \"center\"><img src=\"./figures/monailabel_radiology_spleen_segmentation_OHIF/ohif2.png\" alt=\"drawing\" width=\"900\"/></p>\n",
    "\n",
    "- Click `Select files to upload` and select **all** DICOM files of each subject.\n",
    "- Click `Start the upload` to upload to Orthanc server.\n",
    "- Users can check the uploaded DICOM file for each subject by clicking `All `studies` on the index page.\n",
    "\n",
    "<p align = \"center\"><img src=\"./figures/monailabel_radiology_spleen_segmentation_OHIF/ohif3.png\" alt=\"drawing\" width=\"900\"/></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Start MONAI Label Server\n",
    "\n",
    "The command will start MONAI Label server with all available models. If the user wants to start with a specific model, set the model name in the **--conf models** argument.\n",
    "\n",
    "Such as\n",
    "\n",
    "```bash\n",
    "monailabel start_server --app <path to pathology app> --studies <Orthanc server URL> --conf models <segmentation_spleen>\n",
    "```\n",
    "In the command, ```--app``` specifies the pathology app path, ```--studies``` specifies the datastore URL.\n",
    "\n",
    "By default, the studies argument is specified as: **http://127.0.0.1:8042/dicom-web**\n",
    "\n",
    "**Note:** If you are running a MONAI Label server in this notebook and plan to close the notebook, ensure that you terminate the kernel first. Failing to do so might result in an issue where the port remains occupied from the previous session when you try to restart the server later. To prevent this, always terminate the kernel before closing the notebook."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "!monailabel start_server --app apps/radiology --studies http://127.0.0.1:8042/dicom-web --conf models segmentation_spleen"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Start OHIF Viewer\n",
    "\n",
    "After the monailabel server is up. Users can access OHIF by:\n",
    "\n",
    "**http://127.0.0.1:8000/ohif/**\n",
    "\n",
    "Note: by default is monailabel server is up with port 8000, if port 8000 is occupied by another application, users can specify the port number by `--port <port #>` in monailabel start_server command.\n",
    "Open the OHIF viewer, users can see the subject list as below:\n",
    "\n",
    "<p align = \"center\"><img src=\"./figures/monailabel_radiology_spleen_segmentation_OHIF/ohif4.png\" alt=\"drawing\" width=\"900\"/></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Auto Segmentation with OHIF\n",
    "\n",
    "Below is a step-by-step tutorial on using OHIF for auto-spleen segmentation. Pre-trained model is provided by the MONAI Label.\n",
    "\n",
    "### 6.1 MONAI Label Plugin\n",
    "\n",
    "- MONAI Label plugin is already build-in. Click the subject to annotate, in the labeling page, click `MONAI `Label` button in the top-right corner.\n",
    "\n",
    "<p align = \"center\"><img src=\"./figures/monailabel_radiology_spleen_segmentation_OHIF/ohif5.png\" alt=\"drawing\" width=\"900\"/></p>\n",
    "\n",
    "### 6.2 Run Auto Segmentation\n",
    "\n",
    "- MONAI Label has loaded the pre-trained weights, click **Run** button in the right MONAI Label plugin panel. \n",
    "\n",
    "- Note: the pre-trained models will be downloaded when starting MONAI Label server in the \"**model**\" folder. For instance, in this use case, the downloaded pre-trained model is saved at \"**apps/radiology/model/pretrained_segmentation_spleen.pt**\".\n",
    "\n",
    "Users can monitor the logs in the MONAI Label server terminal.\n",
    "\n",
    "The auto segmentation results can show up as below:\n",
    "\n",
    "<p align = \"center\"><img src=\"./figures/monailabel_radiology_spleen_segmentation_OHIF/ohif6.png\" alt=\"drawing\" width=\"900\"/></p>\n",
    "\n",
    "### 6.3 Edit Annotations and Submit Label\n",
    "\n",
    "- Users can edit the inference labels manually using the OHIF segmentation editor, the tools for adjusting annotation. Once done annotation, click **Submit **Label** on the right plugin panel to save the annotation.\n",
    "\n",
    "- Note: the final annotations will be saved to \"**labels/final**\" folder in the study dataset, for example, in this use case, the ground truth label will be saved to \"**datasets/Task09_Spleen/imagesTr/labels/final**\"\n",
    "\n",
    "<p align = \"center\"><img src=\"./figures/monailabel_radiology_spleen_segmentation_OHIF/ohif7.png\" alt=\"drawing\" width=\"900\"/></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7. Training and Active Learning\n",
    "\n",
    "Active learning and interactive fine-tuning models have highlighted features in MONAI Label, users can train their models anytime when new annotated labels are saved. Click **Update Model** button, MONAI Label server will fetch the saved final ground truth label and fine-tune the prior model. \n",
    "\n",
    "Advanced Feature: users can set training parameters in the `options` panel.\n",
    "\n",
    "<p align = \"center\"><img src=\"./figures/monailabel_radiology_spleen_segmentation_OHIF/ohif8.png\" alt=\"drawing\" width=\"400\"/></p>\n",
    "\n",
    "Users can do auto-segmentation using the latest fine-tuned model. \n",
    "\n",
    "<p align = \"center\"><img src=\"./figures/monailabel_radiology_spleen_segmentation_OHIF/ohif9.png\" alt=\"drawing\" width=\"400\"/></p>\n",
    "\n",
    "The next image is selected, and the already trained images will be marked as labeled data, then won't be selected for the next labeling batch. Active learning strategies such as \"first/random\" will be used for selecting which unlabeled data to fetch. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 8. Conclusion\n",
    "\n",
    "This tutorial demonstrates the basic usage of MONAI Label and OHIF. Use spleen segmentation and radiology app as examples, we provide instructions on how to set up DICOM web server: Orthanc, how to set up OHIF, how to start monailabel server, and basic use cases of auto segmentation, model training and active learning."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  },
  "vscode": {
   "interpreter": {
    "hash": "916dbcbb3f70747c44a77c7bcd40155683ae19c65e1c03b4aa3499c5328201f1"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}