-
Notifications
You must be signed in to change notification settings - Fork 6.8k
[MXNET-116] Inference - ObjectDetector and SSDObjectDetectorExample #10229
Conversation
@@ -0,0 +1,116 @@ | |||
# Single Shot Multi Object Detection using Scala Inference API | |||
|
|||
In this example, you will learn how to use Scala Inference API to import pre-trained Single Shot Multi Object Detection (SSD) MXNet model. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In this example, you will learn how to use Scala Inference API to run Inference on pre-trained Single Shot Multi Object Detection (SSD) MXNet model.
|
||
In this example, you will learn how to use Scala Inference API to import pre-trained Single Shot Multi Object Detection (SSD) MXNet model. | ||
|
||
The pre-trained model is trained on the [Pascal VOC 2012 dataset](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/index.html). The network is a SSD model built on Resnet50 as base network to extract image features. The model is trained to detect the following entities (classes): ['aeroplane', 'bicycle', 'bird', 'boat', 'bottle', 'bus', 'car', 'cat', 'chair', 'cow', 'diningtable', 'dog', 'horse', 'motorbike', 'person', 'pottedplant', 'sheep', 'sofa', 'train', 'tvmonitor']. For more details about the model, you can refer to the [MXNet SSD example](https://github.com/apache/incubator-mxnet/tree/master/example/ssd). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The model is trained on the [Pascal VOC 2012 dataset]
./get_resnet_data.sh | ||
``` | ||
|
||
**Note**: You may need to run `chmod +x get_resnet_data.sh` before running this script. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you still need this if the file already is set to be a executable?
Also you do not need to chmod+x if you running the script as bash ./get_resnet_data.sh
In the pre-trained model, the `input_name` is `data` and shape is `(1, 3, 512, 512)`. | ||
This shape translates to: a batch of `1` image, the image has color and uses `3` channels (RGB), and the image has the dimensions of `512` pixels in height by `512` pixels in width. | ||
|
||
The signature also specifies `image/jpeg` as the expected input type, since this example's image pre-processor only supports the handling of binary JPEG images. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what is this? copy paste error?
|
||
The signature also specifies `image/jpeg` as the expected input type, since this example's image pre-processor only supports the handling of binary JPEG images. | ||
|
||
The signature specifies the output shape is `(1, 6132, 6)`. As with the input, the `1` is the number of images. `6132` is the number of prediction results, and `6` is for the size of each prediction. Each prediction contains the following components: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what is signature?
|
||
|
||
## Infer API Details | ||
This example uses ObjectDetector class provided by MXNet's scala package Infer APIs. It provides methods to load the images, create NDArray out of BufferedImage and run prediction using Classifier and Predictor APIs. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
BufferedImage-> Java BufferedImage
import java.nio.file.{Files, Paths} | ||
|
||
class SSDClassifierExample { | ||
@Option(name = "--model-dir", usage = "the input model directory and prefix of the model") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
--model-path-prefix
class ObjectDetector(modelPathPrefix: String, | ||
inputDescriptors: IndexedSeq[DataDesc]) { | ||
|
||
val classifier: Classifier = getClassifier(modelPathPrefix, inputDescriptors) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
use ImageClassifier instead?
|
||
// Considering 'NCHW' as default layout when not provided | ||
// Else get axis according to the layout | ||
val height = inputShape(if (inputLayout.indexOf('H')<0) 2 else inputLayout.indexOf('H')) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
use these variables from the ImageClassifier.
|
||
val result = objectDetectWithNDArray(IndexedSeq(op), topK) | ||
handler.execute(op.dispose()) | ||
for (ele <- imageBatch) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
imageBatch.foreach(_.dispose())
342a3b1
to
916d1ab
Compare
…pache#10229) * Inference - ObjectDetector and SSDObjectDetectorExample
…pache#10229) * Inference - ObjectDetector and SSDObjectDetectorExample
…pache#10229) * Inference - ObjectDetector and SSDObjectDetectorExample
…pache#10229) * Inference - ObjectDetector and SSDObjectDetectorExample
Description
Add Object Detection Class for MXNet Inference API, also add test and example to it. This code is fully tested with SSD model.
@Roshrini @nswamy @aaronmarkham
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.
Changes