Model Dowload Buttons (#719)

* This creates the component which will populate the Download Tab with Download Buttons.

* Making a place for the download buttons.

* Adding the Model Download Handler allowing the backend to feed the links into the Model View and making slight changes for readablity.

* Getting rid of some of the test code.

* Improve Block usability (#712)

* Use builder pattern for Parameter (#661)

* Make XavierInitializer default value & Improve setInitializer (#664)

* Refactor initialize (#675)

* Remove NDManager on getOutputShapes (#710)

* Removing unnecessary logging messages.

* block factory init commit (#697)

* [DOCS] Fixing TrainingListener documentation (#718)

* Fixing TrainingListener documentation

* Fixing PR reviews

* Fix DJL serving flaky test for mac (#721)

Change-Id: I9eccc84b0c34652e50c5fe5a4fe42f2b82d65a3d

* Fixing all of the nits.

* Getting rid of unnecessary methods.

* update onnxruntime along with String tensor (#724)

* Add profiler doc (#722)

* Resolving some comments.

* Using a better criteria incase multiple models have the same name.

* Fixing the java doc.

* Configure verbose of mxnet extra libraries (#728)

Change-Id: I66d54aa496cccbb9e8c0a89eeaa458605958d9c6

* Added a TODO for using the artifact repo to get the base uri.

* paddlepaddle CN notebook (#730)

* paddlepaddle CN notebook

* install font

Change-Id: I2d749e617b0bf78ecbcd168b82c53a1fab49a2c0

* refactor on name

Change-Id: I9e379eee51ceae16391850b3ba9782acb04c4021

* Refine the text

Co-authored-by: gstu1130 <[email protected]>

* add EI documentation (#733)

* add EI documentation

* fix pmd rules

Change-Id: Ieee5577c26f6df2843781f8f9180de35069a5de3

* allow pytorch stream model loading (#729)

* allow pytorch stream model loading

* updates

Change-Id: Ibc26261b90de673712e90de0d640a8f32f23763e

* add NDList decode from inputStream (#734)

Change-Id: I6a31d8b0b955f2dbb762220b101e3928a34699c1

* Remove memory scope and improve memory management (#695)

The MemoryScope reveals a number of shortcomings within the DJL memory
management. While the MemoryScope is deleted, many of them are fixed as part of
this PR.

First, the NDManager.{attach, detach} were renamed to xxxInternal. This is to
differentiate them from the attach and detach methods that are intended to be used.

There are two new concepts in memory management. An NDResource interface was
created to combine the concepts of managed memory that was used in NDArray and
NDList. It could also be used in more classes in the future. This includes the
getManager, attach, and detach.

Within the NDManager, it gains a second "management convention". The first
convention of normal resources are added to the manager and then closed when the
manager closes. This works for small numbers of things on the NDArray, but not
when operations transitively create. So, the second convention is a
tempResource. Instead of freeing them when the manager is closed, they are
returned to their original manager. This is used to create a temporary scope, do
operations within it, and then the inputs and return value are returned to the
parent while the intermediate work is cleaned. This also matches the concepts of
ownership/borrowing as well.

Using these, a few additional helper methods were created. There is
`NDManager.from(resource)` to ease creation of managers based on a resource.
There is also `scopeManager.ret(returnValue)` to help with returning values
outside of the scopeManager. Lastly, there is a `scopeManager.{temp,}AttachAll`
to attach a number of resources to a manager within a single call.

Using these improvements, the new method were applied to the old locations where
MemoryScope was used as well as an additional case in NDManagerEx.

Also, the old attach methods were altered to be `void`. Because the return
values are no longer used anywhere and are not as necessary in the current
scheme, I figured it would simplify things. It also helps for things like
`NDList.attach` which does not have a single original NDManager when attaching.

Change-Id: I91d109cd14d70fa64fd8fffa0b50d88ab053013e

* Remove erroneous random forest application (#726)

The application was changed to the more accurate softmax_regression (matching
the terminology from the D2L book).

Change-Id: I1f69f005bbe38b125f2709c2988d06c14eebb765

* Minor fixes on duplicated code (#736)

* remove methods that already defined in the NDArrayAdapter

Change-Id: I01cc03a7f5b427bf31c6b3fd8d2136f2a27fe93b

* refactor toString

Change-Id: Iea22b16e1daa9f759b55c1a8b8b85536482e551a

* remove sparse NDArray

Change-Id: Icb44096519775f54cb32cc768c14f49e33dc7ea5

* fix test

Change-Id: Icef580ed77e7bba22864ce44577de3cba51e3e41

Co-authored-by: Jake Lee <[email protected]>
Co-authored-by: Lanking <[email protected]>
Co-authored-by: aksrajvanshi <[email protected]>
Co-authored-by: Frank Liu <[email protected]>
Co-authored-by: Zach Kimberg <[email protected]>

central/src/main/java/ai/djl/serving/central/HttpStaticFileServerInitializer.java

@@ -13,6 +13,7 @@
 package ai.djl.serving.central;
 
 import ai.djl.serving.central.handler.HttpStaticFileServerHandler;
+import ai.djl.serving.central.handler.ModelDownloadHandler;
 import ai.djl.serving.central.handler.ModelMetaDataHandler;
 import io.netty.channel.ChannelInitializer;
 import io.netty.channel.ChannelPipeline;
@@ -54,6 +55,7 @@ public void initChannel(SocketChannel ch) {
         pipeline.addLast(new HttpServerCodec());
         pipeline.addLast(new HttpObjectAggregator(65536));
         pipeline.addLast(new ChunkedWriteHandler());
+        pipeline.addLast(new ModelDownloadHandler());
         pipeline.addLast(new ModelMetaDataHandler());
         pipeline.addLast(new HttpStaticFileServerHandler());
     }

central/src/main/java/ai/djl/serving/central/handler/ModelDownloadHandler.java

@@ -0,0 +1,81 @@
+/*
+ * Copyright 2021 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
+ * with the License. A copy of the License is located at
+ *
+ * http://aws.amazon.com/apache2.0/
+ *
+ * or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
+ * OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions
+ * and limitations under the License.
+ */
+package ai.djl.serving.central.handler;
+
+import ai.djl.repository.zoo.ModelNotFoundException;
+import ai.djl.serving.central.http.BadRequestException;
+import ai.djl.serving.central.responseencoder.HttpRequestResponse;
+import ai.djl.serving.central.utils.ModelUri;
+import ai.djl.serving.central.utils.NettyUtils;
+import io.netty.channel.ChannelHandlerContext;
+import io.netty.channel.SimpleChannelInboundHandler;
+import io.netty.handler.codec.http.FullHttpRequest;
+import io.netty.handler.codec.http.QueryStringDecoder;
+import java.io.IOException;
+import java.util.Collections;
+import java.util.concurrent.CompletableFuture;
+
+/**
+ * A handler to handle download requests from the ModelView.
+ *
+ * @author [email protected]
+ */
+public class ModelDownloadHandler extends SimpleChannelInboundHandler<FullHttpRequest> {
+
+    HttpRequestResponse jsonResponse;
+
+    /** Constructs a ModelDownloadHandler. */
+    public ModelDownloadHandler() {
+        jsonResponse = new HttpRequestResponse();
+    }
+
+    /**
+     * Handles the deployment request by forwarding the request to the serving-instance.
+     *
+     * @param ctx the context
+     * @param request the full request
+     */
+    @Override
+    protected void channelRead0(ChannelHandlerContext ctx, FullHttpRequest request)
+            throws IOException, ModelNotFoundException {
+        QueryStringDecoder decoder = new QueryStringDecoder(request.uri());
+        String modelName = NettyUtils.getParameter(decoder, "modelName", null);
+        String modelGroupId = NettyUtils.getParameter(decoder, "groupId", null);
+        String modelArtifactId = NettyUtils.getParameter(decoder, "artifactId", null);
+        CompletableFuture.supplyAsync(
+                        () -> {
+                            try {
+                                if (modelName != null) {
+                                    return ModelUri.uriFinder(
+                                            modelArtifactId, modelGroupId, modelName);
+                                } else {
+                                    throw new BadRequestException("modelName is mandatory.");
+                                }
+
+                            } catch (IOException | ModelNotFoundException ex) {
+                                throw new IllegalArgumentException(ex.getMessage(), ex);
+                            }
+                        })
+                .exceptionally((ex) -> Collections.emptyMap())
+                .thenAccept(uriMap -> jsonResponse.sendAsJson(ctx, request, uriMap));
+    }
+
+    /** {@inheritDoc} */
+    @Override
+    public boolean acceptInboundMessage(Object msg) {
+        FullHttpRequest request = (FullHttpRequest) msg;
+
+        String uri = request.uri();
+        return uri.startsWith("/serving/models?");
+    }
+}

central/src/main/java/ai/djl/serving/central/http/BadRequestException.java

@@ -0,0 +1,40 @@
+/*
+ * Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
+ * with the License. A copy of the License is located at
+ *
+ * http://aws.amazon.com/apache2.0/
+ *
+ * or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
+ * OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions
+ * and limitations under the License.
+ */
+package ai.djl.serving.central.http;
+
+/** Thrown when a bad HTTP request is received. */
+public class BadRequestException extends IllegalArgumentException {
+
+    static final long serialVersionUID = 1L;
+
+    /**
+     * Constructs an {@code BadRequestException} with the specified detail message.
+     *
+     * @param message The detail message (which is saved for later retrieval by the {@link
+     *     #getMessage()} method)
+     */
+    public BadRequestException(String message) {
+        super(message);
+    }
+
+    /**
+     * Constructs an {@code BadRequestException} with the specified detail message and a root cause.
+     *
+     * @param message The detail message (which is saved for later retrieval by the {@link
+     *     #getMessage()} method)
+     * @param cause root cause
+     */
+    public BadRequestException(String message, Throwable cause) {
+        super(message, cause);
+    }
+}

central/src/main/java/ai/djl/serving/central/http/package-info.java

@@ -0,0 +1,14 @@
+/*
+ * Copyright 2021 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
+ * with the License. A copy of the License is located at
+ *
+ * http://aws.amazon.com/apache2.0/
+ *
+ * or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
+ * OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions
+ * and limitations under the License.
+ */
+/** Contains HTTP codes. */
+package ai.djl.serving.central.http;

central/src/main/java/ai/djl/serving/central/responseencoder/HttpRequestResponse.java

@@ -0,0 +1,123 @@
+/*
+ * Copyright 2021 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
+ * with the License. A copy of the License is located at
+ *
+ * http://aws.amazon.com/apache2.0/
+ *
+ * or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
+ * OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions
+ * and limitations under the License.
+ */
+package ai.djl.serving.central.responseencoder;
+
+import ai.djl.modality.Classifications;
+import ai.djl.modality.Classifications.ClassificationsSerializer;
+import ai.djl.modality.cv.output.DetectedObjects;
+import ai.djl.repository.Metadata;
+import com.google.gson.Gson;
+import com.google.gson.GsonBuilder;
+import com.google.gson.JsonPrimitive;
+import com.google.gson.JsonSerializer;
+import io.netty.buffer.ByteBuf;
+import io.netty.channel.ChannelFuture;
+import io.netty.channel.ChannelFutureListener;
+import io.netty.channel.ChannelHandlerContext;
+import io.netty.handler.codec.http.DefaultFullHttpResponse;
+import io.netty.handler.codec.http.FullHttpRequest;
+import io.netty.handler.codec.http.FullHttpResponse;
+import io.netty.handler.codec.http.HttpHeaderNames;
+import io.netty.handler.codec.http.HttpHeaderValues;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.netty.handler.codec.http.HttpUtil;
+import io.netty.handler.codec.http.HttpVersion;
+import io.netty.util.CharsetUtil;
+import java.lang.reflect.Modifier;
+
+/**
+ * Serialize to json and send the response to the client.
+ *
+ * @author [email protected]
+ */
+public class HttpRequestResponse {
+
+    private static final Gson GSON_WITH_TRANSIENT_FIELDS =
+            new GsonBuilder()
+                    .setDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'")
+                    .setPrettyPrinting()
+                    .excludeFieldsWithModifiers(Modifier.STATIC)
+                    .registerTypeAdapter(Classifications.class, new ClassificationsSerializer())
+                    .registerTypeAdapter(DetectedObjects.class, new ClassificationsSerializer())
+                    .registerTypeAdapter(Metadata.class, new MetaDataSerializer())
+                    .registerTypeAdapter(
+                            Double.class,
+                            (JsonSerializer<Double>)
+                                    (src, t, ctx) -> {
+                                        long v = src.longValue();
+                                        if (src.equals(Double.valueOf(String.valueOf(v)))) {
+                                            return new JsonPrimitive(v);
+                                        }
+                                        return new JsonPrimitive(src);
+                                    })
+                    .create();
+
+    /**
+     * send a response to the client.
+     *
+     * @param ctx channel context
+     * @param request full request
+     * @param entity the response
+     */
+    public void sendAsJson(ChannelHandlerContext ctx, FullHttpRequest request, Object entity) {
+
+        String serialized = GSON_WITH_TRANSIENT_FIELDS.toJson(entity);
+        ByteBuf buffer = ctx.alloc().buffer(serialized.length());
+        buffer.writeCharSequence(serialized, CharsetUtil.UTF_8);
+
+        FullHttpResponse response =
+                new DefaultFullHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK, buffer);
+        response.headers().set(HttpHeaderNames.CONTENT_TYPE, "application/json; charset=UTF-8");
+        boolean keepAlive = HttpUtil.isKeepAlive(request);
+        this.sendAndCleanupConnection(ctx, response, keepAlive);
+    }
+
+    /**
+     * send content of a ByteBuffer as response to the client.
+     *
+     * @param ctx channel context
+     * @param buffer response buffer
+     */
+    public void sendByteBuffer(ChannelHandlerContext ctx, ByteBuf buffer) {
+
+        FullHttpResponse response =
+                new DefaultFullHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK, buffer);
+        response.headers().set(HttpHeaderNames.CONTENT_TYPE, "application/json; charset=UTF-8");
+        this.sendAndCleanupConnection(ctx, response, false);
+    }
+
+    /**
+     * If Keep-Alive is disabled, attaches "Connection: close" header to the response and closes the
+     * connection after the response being sent.
+     *
+     * @param ctx context
+     * @param response full response
+     * @param keepAlive is alive or not
+     */
+    private void sendAndCleanupConnection(
+            ChannelHandlerContext ctx, FullHttpResponse response, boolean keepAlive) {
+        HttpUtil.setContentLength(response, response.content().readableBytes());
+        if (!keepAlive) {
+            // We're going to close the connection as soon as the response is sent,
+            // so we should also make it clear for the client.
+            response.headers().set(HttpHeaderNames.CONNECTION, HttpHeaderValues.CLOSE);
+        }
+
+        ChannelFuture flushPromise = ctx.writeAndFlush(response);
+
+        if (!keepAlive) {
+            // Close the connection as soon as the response is sent.
+            flushPromise.addListener(ChannelFutureListener.CLOSE);
+        }
+    }
+}

central/src/main/java/ai/djl/serving/central/utils/ModelUri.java

@@ -0,0 +1,71 @@
+/*
+ * Copyright 2021 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
+ * with the License. A copy of the License is located at
+ *
+ * http://aws.amazon.com/apache2.0/
+ *
+ * or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
+ * OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions
+ * and limitations under the License.
+ */
+package ai.djl.serving.central.utils;
+
+import ai.djl.Application;
+import ai.djl.repository.Artifact;
+import ai.djl.repository.zoo.Criteria;
+import ai.djl.repository.zoo.ModelNotFoundException;
+import ai.djl.repository.zoo.ModelZoo;
+import java.io.IOException;
+import java.net.URI;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+/** A class to find the URIs when given a model name. */
+public final class ModelUri {
+
+    // TODO: Use the artifact repository to create base URI
+    private static URI base = URI.create("https://mlrepo.djl.ai/");
+
+    private ModelUri() {}
+
+    /**
+     * Takes in a model name, artifactId, and groupId to return a Map of download URIs.
+     *
+     * @param artifactId is the artifactId of the model
+     * @param groupId is the groupId of the model
+     * @param name is the name of the model
+     * @return a map of download URIs
+     * @throws IOException if the uri could not be found
+     * @throws ModelNotFoundException if Model can not be found
+     */
+    public static Map<String, URI> uriFinder(String artifactId, String groupId, String name)
+            throws IOException, ModelNotFoundException {
+        Criteria<?, ?> criteria =
+                Criteria.builder()
+                        .optModelName(name)
+                        .optGroupId(groupId)
+                        .optArtifactId(artifactId)
+                        .build();
+        Map<Application, List<Artifact>> models = ModelZoo.listModels(criteria);
+        Map<String, URI> uris = new ConcurrentHashMap<>();
+        models.forEach(
+                (app, list) -> {
+                    list.forEach(
+                            artifact -> {
+                                for (Map.Entry<String, Artifact.Item> entry :
+                                        artifact.getFiles().entrySet()) {
+                                    URI fileUri = URI.create(entry.getValue().getUri());
+                                    URI baseUri = artifact.getMetadata().getRepositoryUri();
+                                    if (!fileUri.isAbsolute()) {
+                                        fileUri = base.resolve(baseUri).resolve(fileUri);
+                                    }
+                                    uris.put(entry.getKey(), fileUri);
+                                }
+                            });
+                });
+        return uris;
+    }
+}