[MXNET-50] Scala Inference APIs #9678

nswamy · 2018-02-02T02:54:37Z

Description

https://issues.apache.org/jira/browse/MXNET-50
MXNet Scala Inference Library.

Checklist

Essentials

Passed code style checking (make lint)
Changes are complete (i.e. I finished coding on this PR)
All changes have test coverage:
Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
Code is well-documented:
For user-facing API changes, API doc string has been updated.
For new C++ functions in header files, their functionalities and arguments are documented.
For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

Feature1, tests, (and when applicable, API doc)
Feature2, tests, (and when applicable, API doc)

Comments

If this change is a backward incompatible change, why must this change be made.
Interesting edge cases to note here

marcoabreu · 2018-02-02T06:01:24Z

Would it make sense to have this package under the org.apache namespace since it seems like an entirely new aspect? This might allow us to make a quicker transition, away from dmlc.

nswamy · 2018-02-02T16:53:54Z

@marcoabreu, this package needs to pull in the core package under dmlc, so i prefer to keep it under dmlc. I would prefer to have a separate discussion to see how we want to manage versioning of language bindings for MXNet. Currently, It does not make sense to me that a API change in one of the language bindings trigger whole new version for entire MXNet. Also Scala packages have not been published to maven for a while now and the versions are maintained in Maven. we have to address this as well.

nswamy · 2018-02-02T16:55:05Z

Unit Tests are failing for a requirement(number of dimensions in shape match the number of elements in layout) I put into the DataDesc. I will fix the unit tests and update the PR.

nswamy · 2018-02-02T16:55:30Z

@calumleslie @yizhi @lupesko @jesterhazy

yzhliu · 2018-02-05T19:23:38Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/MXNetHandler.scala

+    val task = new Callable[T] {
+      override def call(): T = {
+        // scalastyle:off println
+        println("threadId: %s".format(Thread.currentThread().getId()))


better to use Log

this for testing, removed the print

yzhliu · 2018-02-05T19:25:38Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/MXNetHandler.scala

+
+}
+
+object MXNetSingleThreadHandler extends MXNetOneThreadPerModelHandler {


why do we need an object extends something? any static members to be accessed?

I want to create a singleton object with only thread which will be used by default.

yzhliu · 2018-02-05T19:28:10Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/MXNetHandler.scala

+    }
+  }
+
+  override val executor: ExecutorService = Executors.newFixedThreadPool(10, threadFactory)


make 10 configurable through constructor.

A fixed threadpool of 10 does not enforce the invariant of having a single thread for all MXNet use, which I believe to be the point of this approach. The thread count here should be fixed to 1 (or use Executors.newSingleThreadExecutor).

I want this to be configurable, the value 10 was for testing, made this configurable now.

yzhliu · 2018-02-05T19:33:23Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/MXNetHandler.scala

+      result.get()
+    }
+    catch {
+      case e: ExecutionException => throw e.getCause()


add throw ExecutionException to the signature of this method.
place catch the same line of }

how about other uncaught exceptions?

@yzhliu What's the purpose of adding a @throws annotation? These are normally disregarded in Scala.

It would make sense to document why the executionexception is being unwrapped here (the answer is "so it looks like the code was called inline").

yzhliu · 2018-02-05T19:33:45Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/MXNetHandler.scala

+    if (handlerType == MXNetHandlerType.OneThreadPerModelHandler) {
+      new MXNetOneThreadPerModelHandler
+    }
+    else {


place else the same line of }

yzhliu · 2018-02-05T19:35:21Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/MXNetHandler.scala

+
+  def execute[T](f: => T): T
+
+  val executor: ExecutorService


Arguably this should not be on the trait at all.

I want to be able to expose the executor if at all I find that all NDArray creations have to go throw the same thread.

yzhliu · 2018-02-05T19:38:17Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/package.scala

+import ml.dmlc.mxnet.infer.MXNetHandlerType.MXNetHandlerType
+
+package object infer {
+  private[mxnet] val handlerType: MXNetHandlerType = MXNetHandlerType.SingleThreadHandler


can we change this variable anywhere?

should we make it user-configurable?

I will expose this as a property if I find that MXNet can be called from multiple threads.
, like said in other places I am going with the assumption that MXNet needs to be called from the same thread through out the lifetime of the process otherwise it seg-faults.

CodingCat · 2018-02-06T03:35:24Z

scala-package/core/src/main/scala/ml/dmlc/mxnet/IO.scala

@@ -230,6 +230,8 @@ abstract class DataPack() extends Iterable[DataBatch] {
 // Named data desc description contains name, shape, type and other extended attributes.
 case class DataDesc(name: String, shape: Shape,
                    dtype: DType = Base.MX_REAL_TYPE, layout: String = "NCHW") {
+  require(shape.length == layout.length, "number of dimensions in shape should match the layout")


would you show the current length of both in the error msg?

CodingCat · 2018-02-06T03:36:58Z

scala-package/infer/pom.xml

+                </plugins>
+            </build>
+        </profile>
+    </profiles>


all of the above things can be inherited from parent pom.xml

I wasn't able inherit and make the tests run. the tests needs the jar to be built

CodingCat · 2018-02-06T03:39:06Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/MXNetHandler.scala

+
+import java.util.concurrent._
+
+trait MXNetHandler {


if you are not certain about the stability of the API, you may want to make it private[infer]

CodingCat · 2018-02-06T03:39:22Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/MXNetHandler.scala

+
+}
+
+object MXNetHandlerType extends Enumeration {


if you are not certain about the stability of the API, you may want to make it private[infer]

CodingCat · 2018-02-06T03:49:36Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/MXNetHandler.scala

+
+class MXNetOneThreadPerModelHandler extends MXNetHandler {
+
+  private val threadFactory = new ThreadFactory {


since you are working with Scala, you may want to work with more scala-native concurrency facility, i.e. Future, which provides more elegant way for error handling, etc.

I'll learn about it and let me do that in the next iteration

CodingCat · 2018-02-06T04:01:49Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/PredictBase.scala

+    if (batchSize != 1) {
+      mxNetHandler.execute(mod.bind(inputDescriptors, forTraining = false, forceRebind = true))
+    }
+


extra empty line

Is it standard to not have blank lines, I tend to put blank lines so that it is more readable.

CodingCat · 2018-02-06T04:01:53Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/PredictBase.scala

+
+    for((i, d) <- input.zip(inputDescriptors)) {
+      val shape = d.shape.toVector.patch(from = batchIndex, patch = Vector(1), replaced = 1)
+


extra empty line

please check all the other places

CodingCat · 2018-02-06T04:02:51Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/PredictBase.scala

+  /**
+    * This method will take input as IndexedSeq one dimensional arrays and creates
+    * NDArray needed for inference. The array will be reshaped based on the input descriptors.
+    * @param input: A IndexedSequence of Java one-dimensional array, An IndexedSequence is


CodingCat · 2018-02-06T04:06:25Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/PredictBase.scala

+    for((i, d) <- input.zip(inputDescriptors)) {
+      val shape = d.shape.toVector.patch(from = batchIndex, patch = Vector(1), replaced = 1)
+
+      inputND += mxNetHandler.execute(NDArray.array(i, Shape(shape)))


I am a bit confused here, NDArray.array(i, Shape(shape)) is executed by a new thread?

and it's actually blocking here until the function is finished? why we need to do this?

Calls to MXNet's native library are not safe to call from more than one thread per program execution (unless that has been changed). This is just how MXNet works at present. We described this on the discussion site at one point.

It's actually not entirely known if some calls are safe and some are not but we've found the best stability from single-threading all calls to MXNet.

I am going by Calum's findings here

CodingCat · 2018-02-06T04:07:05Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/package.scala

+import ml.dmlc.mxnet.infer.MXNetHandlerType.MXNetHandlerType
+
+package object infer {
+  private[mxnet] val handlerType: MXNetHandlerType = MXNetHandlerType.SingleThreadHandler


should we make it user-configurable?

calumleslie

I have some feedback on this implementation as well.

calumleslie · 2018-02-12T15:51:58Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/MXNetHandler.scala

+  private val threadFactory = new ThreadFactory {
+
+    override def newThread(r: Runnable): Thread = new Thread(r) {
+      setName(classOf[MXNetOneThreadPerModelHandler].getCanonicalName)


The name here is a little confusing given that it is also used for MXNetSingleThreadHandler.

calumleslie · 2018-02-12T15:54:16Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/MXNetHandler.scala

+
+  type MXNetHandlerType = Value
+  val SingleThreadHandler = Value("MXNetSingleThreadHandler")
+  val OneThreadPerModelHandler = Value("MXNetOneThreadPerModelHandler")


Given that one-thread-per-model is (to the best of my knowledge) currently an unsafe operating mode, is there any point in supporting it just yet?

To be able to support in the future or if I find that its OK to run one-thread-per-model

calumleslie · 2018-02-12T15:55:40Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/MXNetHandler.scala

+      result.get()
+    }
+    catch {
+      case e: ExecutionException => throw e.getCause()


@yzhliu What's the purpose of adding a @throws annotation? These are normally disregarded in Scala.

It would make sense to document why the executionexception is being unwrapped here (the answer is "so it looks like the code was called inline").

calumleslie · 2018-02-12T15:57:15Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/MXNetHandler.scala

+
+  override val executor: ExecutorService = Executors.newFixedThreadPool(10, threadFactory)
+
+  override def execute[T](f: => T): T = {


Do you want to support the recursive case? If you submit a task which in turn wants to submit a task to the MXNet thread, it will deadlock in this arrangement. You can avoid this by checking whether you're already on the managed thread and executing the code inline if so.

I did not encounter this when i unit tested by running on the same thread. However I fixed this based on your implementation.

calumleslie · 2018-02-12T15:58:51Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/MXNetHandler.scala

+    }
+  }
+
+  override val executor: ExecutorService = Executors.newFixedThreadPool(10, threadFactory)


A fixed threadpool of 10 does not enforce the invariant of having a single thread for all MXNet use, which I believe to be the point of this approach. The thread count here should be fixed to 1 (or use Executors.newSingleThreadExecutor).

calumleslie · 2018-02-12T16:00:08Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/PredictBase.scala

+
+    for((i, d) <- input.zip(inputDescriptors)) {
+      require (i.length == d.shape.product/batchSize, "number of elements:" +
+        " %d in the input does not match the shape:%s".format( i.length, d.shape.toString()))


Minor: The use of spaces within parentheses here is inconsistent.

calumleslie · 2018-02-12T16:02:02Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/PredictBase.scala

+    for((i, d) <- input.zip(inputDescriptors)) {
+      val shape = d.shape.toVector.patch(from = batchIndex, patch = Vector(1), replaced = 1)
+
+      inputND += mxNetHandler.execute(NDArray.array(i, Shape(shape)))


Calls to MXNet's native library are not safe to call from more than one thread per program execution (unless that has been changed). This is just how MXNet works at present. We described this on the discussion site at one point.

It's actually not entirely known if some calls are safe and some are not but we've found the best stability from single-threading all calls to MXNet.

calumleslie · 2018-02-12T16:04:26Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/MXNetHandler.scala

+
+  def execute[T](f: => T): T
+
+  val executor: ExecutorService


Arguably this should not be on the trait at all.

CodingCat · 2018-03-07T04:43:17Z

Hi, the community has passed to vote about associating the code changes with JIRA (https://lists.apache.org/thread.html/ab22cf0e35f1bce2c3bf3bec2bc5b85a9583a3fe7fd56ba1bbade55f@%3Cdev.mxnet.apache.org%3E)

We have updated the guidelines for contributors in https://cwiki.apache.org/confluence/display/MXNET/Development+Process, please ensure that you have created a JIRA at https://issues.apache.org/jira/projects/MXNET/issues/ to describe your work in this pull request and include the JIRA title in your PR as [MXNET-xxxx] your title where MXNET-xxxx is the JIRA id

Thanks!

nswamy · 2018-03-09T23:07:58Z

@CodingCat @yzhliu @calumleslie
I have addressed most of your comments and added more code. Unfortunately i force pushed to my branch github is now not showing old comments, If I have missed any let me know.
I have also added unit tests.

@CodingCat
I don't know the specific benefits of using Scala Concurrency features, I think this suffices for now and I'll change it to Scala Concurrency as I learn and improve this in the next iteration.

@calumleslie
For your comment on using checking if it is already on a thread that can run MXNet, I didn't have any issues even when on the same thread. May be I did not understand the problem, can you please elaborate.

yzhliu · 2018-03-13T17:28:31Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/Classifier.scala

+
+  val synset = readSynsetFile(synsetFilePath)
+
+  val handler = MXNetHandler()


can any of the above variable be 'private' or 'private[mxnet]'?

yzhliu · 2018-03-13T17:29:51Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/Classifier.scala

+    * @return IndexedSequence of (Label, Score) tuples.
+    */
+  def classify(input: IndexedSeq[Array[Float]],
+               topK: Option[Int] = None): List[(String, Float)]


Is IndexedSeq better than List? (the return type)

Ok, changed the return type to IndexedSeq instead.

yzhliu · 2018-03-13T18:36:55Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/Classifier.scala

+
+  val synsetFilePath = getSynsetFilePath(modelPathPrefix)
+
+  val synset = readSynsetFile(synsetFilePath)


I think it is better to make synset optional

Classifier is just taking the predictor and using Synset to map the labels, this is the only difference between the predictor/classifier

yzhliu · 2018-03-13T18:47:15Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/Predictor.scala

+    resultND
+  }
+
+  def loadModule(): Module = {


I feel it is better to rely on Symbol and Executor directly. Executor is much cleaner and cannot run into any memory leak before we fix the GC problem.

wouldn't this just be bringing parts of the Module code into this class ?

discussed offline with @yzhliu, Module is necessary to support GPUs.

…ead that created the handler

yzhliu · 2018-03-16T04:22:20Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/Classifier.scala

+
+  protected[mxnet] val synset = readSynsetFile(synsetFilePath)
+
+  protected[mxnet] val handler = MXNetHandler()


can we make them protected[infer]?

yzhliu · 2018-03-16T04:26:49Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/Classifier.scala

+    s.getCanonicalPath
+  }
+
+  def readSynsetFile(synsetFilePath: String): List[String] = {


better to return IndexedSeq since you will do something like synset(sIndx) later.

yzhliu · 2018-03-16T04:35:51Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/Classifier.scala

+    * @param input: Indexed Sequence of NDArrays
+    * @param topK: (Optional) How many top_k(sorting will be based on the last axis)
+    *             elements to return, if not passed returns unsorted output.
+    * @return Traversable Sequence of (Label, Score) tuple, Score will be in the form of NDArray


You mean 'Results will be in the same order as the input NDArray' ? I cannot quite get what 'Score will be in the form of NDArray' means.

this remained from the previous version of the code. thanks for pointing. fixed

sorry, have you fixed? I don't see from my end.

yzhliu · 2018-03-16T04:41:09Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/MXNetHandler.scala

+  override def execute[T](f: => T): T = {
+
+    if (Thread.currentThread() eq creatorThread) {
+      f


better to write f() to distinct from a return of a function variable.

this doesn't work since it does not take any parameters, so I added a return to be more explicit. return statement fails style-check, so i have to keep it the way it is.

This is just how by-name parameters work, it's definitely a bit awkward :(

yzhliu · 2018-03-16T04:41:19Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/MXNetHandler.scala

+    val task = new Callable[T] {
+      override def call(): T = {
+        logger.debug("threadId: %s".format(Thread.currentThread().getId()))
+        f


better to write f() to distinct from a return of a function variable.

yzhliu · 2018-03-16T04:42:53Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/Predictor.scala

+/**
+ * Base Trait for MXNet Predictor classes.
+ */
+private[mxnet] trait PredictBase {


private[infer] ?

yzhliu · 2018-03-16T04:48:09Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/Predictor.scala

+   * This method will take input as IndexedSeq one dimensional arrays and creates
+   * NDArray needed for inference. The array will be reshaped based on the input descriptors.
+   * @param input: A IndexedSequence of Scala one-dimensional array, An IndexedSequence is
+   *             is needed when the model has more than one input/output


Feel confused with 'An IndexedSequence is is needed when the model has more than one input/output'. The IndexedSeq is not about model, it's just an array of input data, one entry for one input sample, right?

this for models that need more than 1 input, a crude example would be a model that takes 2 different images. Are you implying that it can just be another dimension to the same input?

yzhliu · 2018-03-16T04:48:47Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/Predictor.scala

+  protected[mxnet] var batchSize = if (batchIndex != -1) inputDescriptors(0).shape(batchIndex)
+    else 1
+
+  protected[mxnet] var iDescriptors = inputDescriptors


protected[infer] ?

yzhliu · 2018-03-16T04:50:32Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/Predictor.scala

+
+  protected[mxnet] val mxNetHandler = MXNetHandler()
+
+  protected[mxnet] val mod = loadModule()


protected[infer] or even better, protected/private ?

if i don't make it package[infer] private. I won't be able to mock and unit test easily.

yzhliu · 2018-03-16T05:00:31Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/Predictor.scala

+    resultND
+  }
+
+  def loadModule(): Module = {


calumleslie · 2018-03-16T17:00:13Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/MXNetHandler.scala

+  override val executor: ExecutorService =
+    Executors.newFixedThreadPool(numThreads.get, threadFactory)
+
+  private val creatorThread = executor.submit(new Callable[Thread] {


I'm not sure if creatorThread is going to have a meaningful value when the size of the threadpool is not 1.

Given that the size of the thread pool must always be 1 at the moment, maybe it'd be easier to just avoid having to handle multiple threads?

calumleslie · 2018-03-16T17:01:13Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/MXNetHandler.scala

+      try {
+        result.get()
+      } catch {
+        case e: Exception => throw e.getCause()


You should probably only be doing this for ExecutionException. This method can also throw InterruptedException which is not guaranteed to have a cause.

I also think that you should comment explaining why you're unwrapping this exception here.

you might also get other exceptions, I think it makes sense to catch all Exceptions. I will throw InterruptedException as is separately.

I believe this API should only ever throw ExecutionException or InterruptedException but the point was the only type of exception you can safely call "getCause" on is the former. You probably just want all other exceptions to be propagated as is which you can do by just not catching them.

calumleslie · 2018-03-16T17:03:20Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/MXNetHandler.scala

+  val OneThreadPerModelHandler = Value("MXNetOneThreadPerModelHandler")
+}
+
+private[infer] class MXNetThreadPoolHandler(numThreads: Option[Int] = Some(1))


numThreads is an Option[Int] but you always unconditionally dereference it. Is None a valid value? Could you just accept an Int here?

good point, regardless of whether an Option or just a Int, I needed a validation of int > 0. I changed it to a Int and added validation.

calumleslie · 2018-03-16T17:06:29Z

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/MXNetHandler.scala

+
+  private val threadFactory = new ThreadFactory {
+
+    override def newThread(r: Runnable): Thread = new Thread(r) {


This will create all threads with the same name, which will make them difficult (or in some contexts impossible) to tell apart. If you're going to continue to support multiple threads consider incrementing a counter whenever you vend a thread and appending the number to the name (this is the behaviour of Guava's ThreadFactoryBuilder for example).

calumleslie · 2018-03-16T17:09:54Z

@nswamy Sorry it took me a while to get back I've been off ill. Also I don't really use GitHub so often miss the notifications! I've added some more comments on some of the implementation here.

For your comment on using checking if it is already on a thread that can run MXNet, I didn't have any issues even when on the same thread. May be I did not understand the problem, can you please elaborate.

It looks like you fixed this with the creatorThread stuff :)

* Scala Inference APIs * fix unit tests for shape.length == layout.length in DataDesc * make ThreadPoolHandler of size 1 * Rename PredictBase to Predictor * change classify output from List to IndexedSeq * modify MXNetHandler to check if the task is executing on the same thread that created the handler * add argument epoch for Predictor/Classifier

nswamy requested a review from yzhliu February 2, 2018 02:54

yzhliu suggested changes Feb 5, 2018

View reviewed changes

CodingCat suggested changes Feb 6, 2018

View reviewed changes

calumleslie suggested changes Feb 12, 2018

View reviewed changes

nswamy force-pushed the scala-package branch from 7fb5992 to dad1436 Compare February 28, 2018 19:04

nswamy force-pushed the scala-package branch 2 times, most recently from 72e8bc6 to 85545fa Compare March 9, 2018 21:54

nswamy changed the title ~~[First cut] Scala Inference APIs~~ [MXNET-50] Scala Inference APIs Mar 9, 2018

Roshrini mentioned this pull request Mar 9, 2018

[MXNET-53]Image classifier for scala-infer package #10054

Merged

7 tasks

yzhliu suggested changes Mar 13, 2018

View reviewed changes

nswamy force-pushed the scala-package branch from a788317 to caa9765 Compare March 14, 2018 19:09

nswamy added 13 commits March 14, 2018 18:47

Scala Inference APIs

ffb25a7

fix unit tests for shape.length == layout.length in DataDesc

fbb7167

update comments

d0a41e5

make ThreadPoolHandler of size 1

a320d76

add Classifier Implmentation

c0dc0aa

unit tests(in progress) - not building

4c4c5ad

unit tests(wip)

94dfd0f

add Predictor UT

c822b81

add Classifier UT

3c24dc0

add Classifier UT

203aaec

Rename PredictBase to Predictor

ee9cf36

uncomment test Ignore

3186528

fix ut

082cddb

nswamy added 3 commits March 14, 2018 18:47

address review comments

ada520c

change classify output from List to IndexedSeq

6245960

modify MXNetHandler to check if the task is executing on the same thr…

95f1ab8

…ead that created the handler

nswamy force-pushed the scala-package branch 2 times, most recently from 9ca4156 to c370736 Compare March 16, 2018 02:05

add argument epoch for Predictor/Classifier

5e44330

nswamy force-pushed the scala-package branch from c370736 to 5e44330 Compare March 16, 2018 03:56

yzhliu suggested changes Mar 16, 2018

View reviewed changes

address review comments

95657b1

nswamy force-pushed the scala-package branch from a879e1c to 95657b1 Compare March 16, 2018 14:40

calumleslie suggested changes Mar 16, 2018

View reviewed changes

nswamy force-pushed the scala-package branch 2 times, most recently from 37082d3 to ac2fe86 Compare March 16, 2018 21:18

address comments

37caebe

nswamy force-pushed the scala-package branch from ac2fe86 to 37caebe Compare March 16, 2018 21:36

yzhliu approved these changes Mar 16, 2018

View reviewed changes

nswamy merged commit b95ae7c into apache:master Mar 16, 2018


		}

		object MXNetSingleThreadHandler extends MXNetOneThreadPerModelHandler {


		class MXNetOneThreadPerModelHandler extends MXNetHandler {

		private val threadFactory = new ThreadFactory {


		for((i, d) <- input.zip(inputDescriptors)) {
		val shape = d.shape.toVector.patch(from = batchIndex, patch = Vector(1), replaced = 1)


		override val executor: ExecutorService = Executors.newFixedThreadPool(10, threadFactory)

		override def execute[T](f: => T): T = {


		val synset = readSynsetFile(synsetFilePath)

		val handler = MXNetHandler()


		val synsetFilePath = getSynsetFilePath(modelPathPrefix)

		val synset = readSynsetFile(synsetFilePath)


		protected[mxnet] val synset = readSynsetFile(synsetFilePath)

		protected[mxnet] val handler = MXNetHandler()


		protected[mxnet] val mxNetHandler = MXNetHandler()

		protected[mxnet] val mod = loadModule()


		private val threadFactory = new ThreadFactory {

		override def newThread(r: Runnable): Thread = new Thread(r) {

[MXNET-50] Scala Inference APIs #9678

[MXNET-50] Scala Inference APIs #9678

Conversation

nswamy commented Feb 2, 2018 • edited Loading

Description

Checklist

Essentials

Changes

Comments

marcoabreu commented Feb 2, 2018

nswamy commented Feb 2, 2018

nswamy commented Feb 2, 2018

nswamy commented Feb 2, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

calumleslie left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

CodingCat commented Mar 7, 2018

nswamy commented Mar 9, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yzhliu Mar 16, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nswamy Mar 16, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nswamy commented Feb 2, 2018 •

edited

Loading

nswamy commented Feb 2, 2018 •

edited

Loading

yzhliu Mar 16, 2018 •

edited

Loading

nswamy Mar 16, 2018 •

edited

Loading

nswamy Mar 16, 2018 •

edited

Loading