jupyter/object_detection_with_model_zoo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Object detection with model zoo model\n",
    "\n",
    "In this tutorial, you learn how to use a built-in model zoo model (SSD) to achieve an [object detection](https://en.wikipedia.org/wiki/Object_detection) task.\n",
    "\n",
    "## Preparation\n",
    "\n",
    "This tutorial requires the installation of Java Kernel. To install Java Kernel, see the [README](https://github.com/deepjavalibrary/djl/blob/master/jupyter/README.md)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "// %mavenRepo snapshots https://oss.sonatype.org/content/repositories/snapshots/\n",
    "\n",
    "%maven ai.djl:api:0.19.0\n",
    "%maven ai.djl.mxnet:mxnet-engine:0.19.0\n",
    "%maven ai.djl.mxnet:mxnet-model-zoo:0.19.0\n",
    "%maven org.slf4j:slf4j-simple:1.7.32"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import ai.djl.modality.cv.*;\n",
    "import ai.djl.modality.cv.output.*;\n",
    "import ai.djl.modality.cv.util.*;\n",
    "import ai.djl.mxnet.zoo.*;\n",
    "import ai.djl.repository.zoo.*;\n",
    "import ai.djl.training.util.*;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1: Load image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "var img = ImageFactory.getInstance().fromUrl(\"https://resources.djl.ai/images/dog_bike_car.jpg\");\n",
    "img.getWrappedImage()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: Load model zoo model\n",
    "\n",
    "In this example, you load a SSD (Single Shot MultiBox Detector) model from the MXNet model zoo.\n",
    "For more information about model zoo, see the [Model Zoo Documentation](https://github.com/deepjavalibrary/djl/blob/master/docs/model-zoo.md) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "var criteria = Criteria.builder()\n",
    "    .setTypes(Image.class, DetectedObjects.class)\n",
    "    .optArtifactId(\"ssd\")\n",
    "    .optProgress(new ProgressBar())\n",
    "    .build();\n",
    "var model = criteria.loadModel();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: Create Predictor and detect an object in the image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "var detections = model.newPredictor().predict(img);\n",
    "\n",
    "detections"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Check detected result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "img.drawBoundingBoxes(detections);\n",
    "img.getWrappedImage()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "Using the model zoo model provided, you can run inference with just the following lines of code:\n",
    "\n",
    "```\n",
    "var img = ImageFactory.getInstance().fromUrl(\"https://resources.djl.ai/images/dog_bike_car.jpg\");\n",
    "var criteria = Criteria.builder()\n",
    "    .setTypes(Image.class, DetectedObjects.class)\n",
    "    .optArtifactId(\"ssd\")\n",
    "    .build();\n",
    "var model = criteria.loadModel();\n",
    "var detections = model.newPredictor().predict(img);\n",
    "```\n",
    "\n",
    "You can find full SsdExample source code [here](https://github.com/deepjavalibrary/djl/blob/master/examples/docs/object_detection.md).\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Java",
   "language": "java",
   "name": "java"
  },
  "language_info": {
   "codemirror_mode": "java",
   "file_extension": ".jshell",
   "mimetype": "text/x-java-source",
   "name": "Java",
   "pygments_lexer": "java",
   "version": "14.0.2+12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}