Mito_Catcher.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1 style=\"text-align:center;\">Mito Catcher <span style=\"text-align:center;font-size: 0.5em;\">0.4.1</span></h1>\n",
    "<h2 style=\"text-align:center;\">Mito Hacker Toolkit <i style=\"font-size: 0.5em;\">0.7.1</i></h2>\n",
    "<h3 style=\"text-align:center;\">Kashatus Lab @ UVA</h3>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Welcome to Mito Catcher\n",
    "#### Mito Catcher is part of Mito Hacker toolkit that enables you to segment and extract the mitochondrial network of the cells from fluorescently labeled single-cell images.\n",
    "\n",
    "This Jupyter notebook provides you with step-by-step directions to segment the mitochondrial network in your cells."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 0) Import necessary libraries\n",
    "Please check the requirements in the Readme file."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Just run the following block of code, you are not expected to enter any data here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Base Libraries\n",
    "import os\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from matplotlib.gridspec import GridSpec\n",
    "import cv2\n",
    "\n",
    "#Interaction\n",
    "import ipywidgets as widgets\n",
    "from IPython.display import display\n",
    "from ipywidgets import interact, interactive, fixed, interact_manual, IntSlider, Checkbox, FloatSlider, Dropdown\n",
    "from IPython.display import clear_output\n",
    "\n",
    "#Core Functions\n",
    "import cemia55s as cemia\n",
    "\n",
    "layout = widgets.Layout(width='90%')\n",
    "style = {'description_width': 'initial'}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1) Choose your plan\n",
    "Choose your plan here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plan = widgets.Dropdown(\n",
    "    options=['I am using Mito Catcher to segment the cells that I previously isolated by Cell Catcher.', 'I have my own set of single cell images.'],\n",
    "    value='I am using Mito Catcher to segment the cells that I previously isolated by Cell Catcher.',\n",
    "    description='Your Plan:',\n",
    "    layout=layout,style=style)\n",
    "display(plan)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2) Locate and sample files\n",
    "\n",
    "<br>\n",
    "<details>\n",
    "<summary><span style=\"font-size:16px;font-weight: bold; color:red\">What should I do next? What should I enter in the input box of the next cell? (click here to expand)</span></summary>\n",
    "    \n",
    "#### <span style=\"color:red;\">You need to interact with the next cell: </span>  Please run the next cell, a box will appear. Enter the relative/absolute address of the folder that contains your images inside the box, then press enter.  (read the notes below). \n",
    "\n",
    "#### <span style=\"color:red;\">Examples: </span>\n",
    "* Relative Address\n",
    "    * Use . if the images are in the same folder as this file\n",
    "    * If your folder of the images (my_folder_of_images) is located in the same directory as this file, you should enter: my_folder_of_images\n",
    "* Absolute Address\n",
    "    *If your images are located on Desktop\n",
    "        * Mac: you should enter: /Users/username/Desktop/my_folder_of_images\n",
    "        * Windows: you should enter: C:\\Users\\username\\Desktop\\my_folder_of_images\n",
    "\n",
    "#### <span style=\"color:red;\">Notes: </span>\n",
    "* It is preferred to have the folder of your images in the same folder as the current file that you are running\n",
    "* After you entered the desired value in the box, you should press the enter key.\n",
    "* If you have previously used Cell Catcher to isolate individual cells, and you want to analyze those images, you should enter the address where your original images (Multi-cell images) are located. This is the same address you used in Cell Catcher.\n",
    "    * Example: If you have previously used Cell Catcher to isolate individual cells in a folder named \"test\", you should enter \"test\" as the address to your files. \n",
    "        * Cell Catcher has already created and filled the appropriate set of subfolders in the \"test\" directory, and Mito Miner will locate the files automatically.\n",
    "* If you want to use Mito Miner with your own single cell images, you should enter the address where your files are located, and Mito Miner will guide you how to relocate your files.\n",
    "    \n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "address,file_list = cemia.address()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3) Satisfy basic requirements based on your plan."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cemia.check_dir_requirements_miner(address)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "counter, temp_address = cemia.check_requirements_miner(plan.value,address)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cemia.confirm_requirements_miner(plan.value, address, counter)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4) Segment Sample Cells\n",
    "\n",
    "### 4-1) Selecting a filtering method\n",
    "\n",
    "<br>\n",
    "<details>\n",
    "<summary><span style=\"font-size:16px;font-weight: bold; color:red\">Additional info</span></summary>\n",
    "\n",
    "##### There are two different method available to help you to segment your mitochondrial network. You can test them and then decide which one works for your dataset better.\n",
    "#### <span style=\"color:red;\">Availabe Methods: </span>\n",
    "#### <b>Let It Go:<b> Better option for images with very low background intensity.\n",
    "#### <b>Not One of Us:</b> Signal separation based on the distribution of background intensity in the nucleus area.\n",
    "#### <span style=\"color:red;\">Notes: </span>\n",
    "\n",
    "###### Once you selected a method, go and run the next cell. Rerunning the same cell may reset your selection.\n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "filter_method = widgets.Dropdown(options=['Let It Go','Not One of Us'],description='Filter:',style=style, layout=layout)\n",
    "print('\\nLet It Go: Better option for images with low background intensity.')\n",
    "print('\\nNot One of Us: Signal separation based on the distribution of background intensity in the nucleus area.')\n",
    "display(filter_method)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4-2) Tweaking the selected filter parameters for accurate segmentation\n",
    "\n",
    "<br>\n",
    "<details>\n",
    "<summary><span style=\"font-size:16px;font-weight: bold; color:red\">List of parameters you are adjusting here (click to expand the list)</span></summary>\n",
    "    \n",
    "<p><b>Dark Image:</b> This option only visually enhances the dark images and has no effect on the analysis. Selecting this option makes images 3X brighter, and it is useful for comparison when signal level is low, and it is hard to visualize the original cells.</p>\n",
    "<p><b>Filter Strength:</b> Determines the strength of the background removal.</p>\n",
    "<p><b>Filter Harshness:</b> Determines if the background removal is harshly applied to a small group of images, or whether it is more softly applied to a wider range of images.</p>\n",
    "<p></p>\n",
    "<p><b>Adaptive Options</b></p>\n",
    "<p>Settings for the adaptive method are effective only if the adaptive method is selected.</p>\n",
    "<p><b>Adaptive Filter Tile Size:</b> The size of the tile over which the adaptive function samples data and compares with the reference intensity.</p>\n",
    "<p><b>Adaptive Filter Power:</b> The strength of the adaptive function in correcting the threshold.</p>\n",
    "<p><b>Adaptive Filter Footprint:</b> Determines whether the correction factor should affect a wide, or short range of images with different background intensities.</p>\n",
    "<p><b>Remove Debris:</b> Tries to remove noisy objects created in the image as a result of using different filters</p>\n",
    "\n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "#Initializing model parameters\n",
    "params = cemia.mito_miner_initial_params\n",
    "\n",
    "tmp1 = []\n",
    "tmp2 = []\n",
    "sample_files = cemia.check_files_before_filter(temp_address)\n",
    "sample_files = cemia.random_files(temp_address,sample_files,10)\n",
    "\n",
    "print('Selected Method: ',filter_method.value)\n",
    "\n",
    "if filter_method.value=='Not One of Us':\n",
    "    @interact(dark=Checkbox(value=False,description='Dark Image',layout = layout, style=style),\n",
    "              thresh9=IntSlider(value=2, min=1, max=5,step=1, description='Filter Harshness:',continuous_update=False,layout = layout, style=style),\n",
    "              thresh10=IntSlider(value=1, min=1, max=10,step=1, description='Filter Strength:',continuous_update=False,layout = layout, style=style),\n",
    "              adapt=Checkbox(value=False,description='Adaptive Filter Threshold',layout = layout, style=style),\n",
    "              window=Dropdown(options=[8,16,32,64,128,256,512], value=16, description='Adaptive Filter Tile Size',layout = layout, style=style),\n",
    "              equalizer=FloatSlider(value=0.15, min=0.1, max=0.5,step=0.05, description='Adaptive Filter Power:',continuous_update=False,layout = layout, style=style),\n",
    "              steepness=FloatSlider(value=0.5, min=0.25, max=1.5,step=0.25, description='Adaptive Filter Footprint:',continuous_update=False,layout = layout, style=style),\n",
    "              debries=Checkbox(value=True,description='Remove Debris',layout = layout, style=style))\n",
    "    \n",
    "    def filter_selection3(dark,thresh10, thresh9,adapt,window, equalizer,steepness, debries):\n",
    "        \n",
    "        #Updating parameters based on user interaction\n",
    "        params['thresh_median_mask'].append(thresh9)\n",
    "        params['thresh_median_mask2'].append(thresh10)\n",
    "        params['adaptive_th'].append(adapt)\n",
    "        params['window_size_th'].append(window)\n",
    "        params['steepness_th'].append(steepness)\n",
    "        params['equalizer_th'].append(equalizer)\n",
    "        params['remove_debries_th'].append(debries)\n",
    "\n",
    "        for file in sample_files:\n",
    "            try:\n",
    "                pic2 = plt.imread(os.path.join(temp_address, file))\n",
    "\n",
    "                print('Strength: ', thresh10)\n",
    "                print('Harshness: ', thresh9)\n",
    "                print('File: ',os.path.join(temp_address, file))\n",
    "\n",
    "                _,_, binarized, blue_mask = cemia.behind_the_moon_filter(pic2, 60,2.5,1,bg_harshness=-0.5,sig_harshness=params['thresh_median_mask'][-1],sig_strength=params['thresh_median_mask2'][-1], cleanup=False, method=2, adaptive=params['adaptive_th'][-1], window_size=params['window_size_th'][-1],equalizer=params['equalizer_th'][-1],steepness=params['steepness_th'][-1], remove_debries=params['remove_debries_th'][-1])\n",
    "                params['out'].append(binarized)\n",
    "                cemia.filter_plot_mito_miner(pic2,binarized,dark)\n",
    "\n",
    "            except:\n",
    "                print('\\n{} is not a valid image file, trying the next file.'.format(file))\n",
    "                pass\n",
    "                        \n",
    "elif filter_method.value=='Let It Go':\n",
    "    @interact(dark=Checkbox(value=False,description='Dark Image',layout = layout, style=style),\n",
    "              thresh11=FloatSlider(value=-0.5, min=-1.5, max=1,step=0.1, description='Filter Harshness:',continuous_update=False,layout = layout, style=style),\n",
    "              adapt=Checkbox(value=False,description='Adaptive Filter Threshold',layout = layout, style=style),\n",
    "              window=Dropdown(options=[8,16,32,64,128,256,512], value=16, description='Adaptive Filter Tile Size',layout = layout, style=style),\n",
    "              equalizer=FloatSlider(value=0.15, min=0.1, max=0.5,step=0.05, description='Adaptive Filter Power',continuous_update=False,layout = layout, style=style),\n",
    "              steepness=FloatSlider(value=0.5, min=0.25, max=1.5,step=0.25, description='Adaptive Filter Footprint',continuous_update=False,layout = layout, style=style),\n",
    "              debries=Checkbox(value=True,description='Remove Debries',layout = layout, style=style))        \n",
    "    \n",
    "    def filter_selection3(dark,thresh11,adapt,window, equalizer,steepness, debries):\n",
    "\n",
    "        params['thresh_median_signal'].append(thresh11)\n",
    "        params['adaptive_th'].append(adapt)\n",
    "        params['window_size_th'].append(window)\n",
    "        params['steepness_th'].append(steepness)\n",
    "        params['equalizer_th'].append(equalizer)\n",
    "        params['remove_debries_th'].append(debries)\n",
    "\n",
    "        for file in sample_files:\n",
    "           \n",
    "            try:\n",
    "                pic2 = plt.imread(os.path.join(temp_address, file))\n",
    "                print('Harshness: ', thresh11)\n",
    "                print('File: ',os.path.join(temp_address, file))\n",
    "\n",
    "                _,_,binarized, blue_mask = cemia.behind_the_moon_filter(pic2, 60,2.5,1,bg_harshness=params['thresh_median_signal'][-1],sig_harshness=2,sig_strength=1, cleanup=False, method=1, adaptive=params['adaptive_th'][-1], window_size=params['window_size_th'][-1],equalizer=params['equalizer_th'][-1],steepness=params['steepness_th'][-1], remove_debries=params['remove_debries_th'][-1])\n",
    "                params['out'].append(binarized)\n",
    "                cemia.filter_plot_mito_miner(pic2,binarized,dark)\n",
    "\n",
    "            except:\n",
    "                print('\\n{} is not a valid image file, trying the next file.'.format(file))\n",
    "                pass         "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Export all the settings\n",
    "##### This part generates a file that should be copied to the same folder, where the images reside."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('The following settings will be used to analyze all the images!')\n",
    "print('**************************************************************')\n",
    "params_pd = {}\n",
    "for k in params:\n",
    "    if k != 'out':\n",
    "        try:\n",
    "            params_pd[k] = params[k][-1]\n",
    "            #print(f'{k}: {params_pd[k]}')\n",
    "        except:\n",
    "            pass\n",
    "    \n",
    "params_pd = pd.DataFrame(params_pd, index=[0])\n",
    "params_pd.to_csv(os.path.join(address ,'mito_miner_params.csv'), index=False)\n",
    "\n",
    "cemia.report_mito_miner(filter_method.value,params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4) Processing all the cells"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "final_folder = os.path.join(address, 'output','processed','single_cells_binary')\n",
    "try:\n",
    "    os.makedirs(final_folder)\n",
    "except FileExistsError:\n",
    "    pass\n",
    "\n",
    "cell_list = os.listdir(temp_address)\n",
    "\n",
    "if filter_method.value=='Not One of Us':\n",
    "  \n",
    "        for file in cell_list:\n",
    "            fullpath_input = file\n",
    "            abspath = address\n",
    "                         \n",
    "            try:\n",
    "                pic2 = plt.imread(os.path.join(temp_address, file))\n",
    "                print('\\n\\nProcessing File: ',os.path.join(temp_address, file))\n",
    "\n",
    "                img,_,binarized, blue_mask = cemia.behind_the_moon_filter(pic2, 60,2.5,1,bg_harshness=-0.5,sig_harshness=params['thresh_median_mask'][-1],sig_strength=params['thresh_median_mask2'][-1], cleanup=False, method=2, adaptive=params['adaptive_th'][-1], window_size=params['window_size_th'][-1],equalizer=params['equalizer_th'][-1],steepness=params['steepness_th'][-1], remove_debries=params['remove_debries_th'][-1])\n",
    "                cv2.imwrite(final_folder + '/' + file[:file.rfind('.')]+'_mask.tif', binarized)\n",
    "\n",
    "                cv2.imwrite(final_folder + '/' + file, cv2.cvtColor(pic2, cv2.COLOR_RGB2BGR))\n",
    "\n",
    "            except:\n",
    "                print('\\n{} is not a valid image file, trying the next file.'.format(file))\n",
    "                pass\n",
    "\n",
    "elif filter_method.value=='Let It Go':\n",
    "\n",
    "    for file in cell_list:\n",
    "        fullpath_input = file\n",
    "        abspath = address\n",
    "\n",
    "        try:\n",
    "            pic2 = plt.imread(os.path.join(temp_address, file))\n",
    "\n",
    "            print('\\n\\nProcessing File: ',os.path.join(temp_address, file))\n",
    "\n",
    "            img,_,binarized, blue_mask = cemia.behind_the_moon_filter(pic2, 60,2.5,1,bg_harshness=params['thresh_median_signal'][-1],sig_harshness=2,sig_strength=1, cleanup=False, method=1, adaptive=params['adaptive_th'][-1], window_size=params['window_size_th'][-1],equalizer=params['equalizer_th'][-1],steepness=params['steepness_th'][-1], remove_debries=params['remove_debries_th'][-1])\n",
    "            cv2.imwrite(final_folder + '/' + file[:file.rfind('.')]+'_mask.tif', binarized)\n",
    "            cv2.imwrite(final_folder + '/' + file, cv2.cvtColor(pic2, cv2.COLOR_RGB2BGR))\n",
    "\n",
    "        except:\n",
    "            print('\\n{} is not a valid image file, trying the next file.'.format(file))\n",
    "            pass\n",
    "\n",
    "print('\\nDone with the filtering!\\n')\n",
    "print('\\nYou are all set!')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The End!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "position": {
    "height": "458.4px",
    "left": "1464.2px",
    "right": "20px",
    "top": "132px",
    "width": "350px"
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}