Skip to content

Home

Jump to bottom
Animesh Singh edited this page Feb 19, 2018 · 15 revisions

Welcome to the FfDL wiki!

Short Name

Create and Deploy a Deep Learning platform on Kubernetes

Short Description

Deploy a Deep Learning Platform on Kubernetes, offering TensorFlow, Caffe, PyTorch etc. as a Service.

Offering Type

Cognitive, Cloud

Introduction

This Code Pattern shows how to build your own data set and train a model for image classification. The models are available in TensorFlow and are run on a Kubernetes cluster. The demonstration code pulls data and labels from the New York Metropolitan Museum of Art website and Google BigQuery. IBM Cloud container service provides the Kubernetes cluster. Developers can modify the code to build different image data sets and select from a collection of public model such as Inception, VGG, Resnet, Alexnet, Mobilenet, etc.

Author

By Animesh Singh, Scott Boag, Tommy Li, Waldemar Hummer

Code

https://github.com/IBM/FfDL

Demo

N/A

Video

https://youtu.be/I-8xmMxo-RQ

Overview

The ability to process and understand a digital image has wide application and has been an active area of research and development for many years. Recent advances in Deep Learning have brought significant improvements to image recognition and classification, to the degree that many neural network models are now available publicly that offer state-of-the-art performance. If your application requires recognizing some characteristics of an image, you can leverage one of these models to train and deploy a neural network to serve your application.

To explain this process in practice, in this Code Pattern we will use Deep Learning to train an image classification model. For the specific, we will use data from the art collection at the New York Metropolitan Museum of Art and the metadata from Google BigQuery. We will use the Inception model implemented in TensorFlow and we will run the training on a Kubernetes cluster. We will save the trained model and load it later to perform inference. To use the model, we provide as input a picture of a painting and the model will return the likely culture, for instance Italian Florence art. You can adapt the data by choosing some other attributes to classify the art collection, for instance author, time period, etc. You can choose an entirely different source of data, or a different category for classification along with different ways to create the labels. You can also choose other models such as VGG, Resnet, Alexnet, Mobilenet, etc.

Depending on the compute resource available, you can choose the number of images to train, the number of classes to use, etc. For the purpose of showing the full working process, we will select a small set of images and a small number of classes to allow the training to complete within a reasonable amount of time. With a large data set, the training may take days or weeks.

When the reader has completed this Code Pattern, they will understand how to:

  • Collect and process the data for Deep Learning in TensorFlow
  • Configure and deploy TensorFlow to run on a Kubernetes cluster
  • Train an advanced image classification Neural Network
  • Use TensorBoard to visualize and understand the training process

Flow

  1. Inspect the available attributes in the Google BigQuery database for the Met art collection
  2. Create the labeled dataset using the attribute selected
  3. Select a model for image classification from the set of available public models and deploy to IBM Cloud
  4. Run the training on Kubernetes, optionally using GPU if available
  5. Save the trained model and logs
  6. Visualize the training with TensorBoard
  7. Load the trained model in Kubernetes and run an inference on a new art drawing to see the classification

Included components

Featured technologies

Blog

Visualizing High Dimension data for Deep Learning

Links

Clone this wiki locally