Scala 2.11 binding for version 1.5.1 throws memory leak warnings

[cosmincatalin]

I am using the Scala 2.11 binding in my project (🤞 for #18655) and keep getting memory leak warnings. This happens for both the MacOS and Linux builds.

It is not clear to me if these warnings are valid or not, since I would assume the issue was fixed a long time ago (#10436 and some others).

//libraryDependencies += "org.apache.mxnet" % "mxnet-full_2.11-linux-x86_64-cpu" % "1.5.1"
libraryDependencies += "org.apache.mxnet" % "mxnet-full_2.11-osx-x86_64-cpu" % "1.5.1"

020-07-08 15:46:48 WARN  WarnIfNotDisposed:70 - LEAK: An instance of class org.apache.mxnet.NDArray was not disposed. Creation point of this resource was:
	java.lang.Thread.getStackTrace(Thread.java:1559)
	org.apache.mxnet.WarnIfNotDisposed$class.$init$(WarnIfNotDisposed.scala:52)
	org.apache.mxnet.NDArray.<init>(NDArray.scala:744)
	org.apache.mxnet.NDArray.at(NDArray.scala:927)
	org.apache.mxnet.NDArray$$anonfun$org$apache$mxnet$NDArray$$buildStringHelper$1.apply$mcVI$sp(NDArray.scala:873)
	scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160)
	org.apache.mxnet.NDArray.org$apache$mxnet$NDArray$$buildStringHelper(NDArray.scala:872)
	org.apache.mxnet.NDArray$$anonfun$org$apache$mxnet$NDArray$$buildStringHelper$1.apply$mcVI$sp(NDArray.scala:873)
	scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160)
	org.apache.mxnet.NDArray.org$apache$mxnet$NDArray$$buildStringHelper(NDArray.scala:872)
	org.apache.mxnet.NDArray$$anonfun$org$apache$mxnet$NDArray$$buildStringHelper$1.apply$mcVI$sp(NDArray.scala:873)
	scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160)
	org.apache.mxnet.NDArray.org$apache$mxnet$NDArray$$buildStringHelper(NDArray.scala:872)
	org.apache.mxnet.NDArray.toString(NDArray.scala:846)
	java.lang.String.valueOf(String.java:2994)
	scala.collection.mutable.StringBuilder.append(StringBuilder.scala:200)
	scala.collection.TraversableOnce$$anonfun$addString$1.apply(TraversableOnce.scala:364)
	scala.collection.Iterator$class.foreach(Iterator.scala:891)
	scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
	scala.collection.TraversableOnce$class.addString(TraversableOnce.scala:357)
	scala.collection.AbstractIterator.addString(Iterator.scala:1334)
	scala.collection.TraversableOnce$class.mkString(TraversableOnce.scala:323)
	scala.collection.AbstractIterator.mkString(Iterator.scala:1334)
	scala.runtime.ScalaRunTime$._toString(ScalaRunTime.scala:166)
	com.audienceproject.User.toString(User.scala:6)
        ....

I run my code with -Dmxnet.traceLeakedObjects=true so that I can get the verbose trace. The warnings are coming up from just about anywhere I use NDArray, I'm not even doing any inference or training at this point.

[johsbk]

@cosmincatalin this is because your NDarrays are not deallocated automatically, you can use ResourceScope, like this:
ResourceScope.using(new ResourceScope()) { val bareArray = row.getAs[Seq[Double]](fieldName).map(_.toFloat).toArray val shape = schema.getShape(fieldName) val transposedShape = shape.reverse val expectedSize = shape.product val paddedArray = bareArray.padTo(expectedSize, 0f) NDArray.array(paddedArray, Shape(transposedShape), ctx).T.reshape(Shape(List(1, 1) ++ shape: _*)).toArray }
to automatically close NDarrays

[cosmincatalin]

Great stuff @johsbk