-
Notifications
You must be signed in to change notification settings - Fork 92
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[API][v0.3] Connect HeteroCL with Keras Models via Relay (#142)
This PR implements a pass between Keras and HeteroCL. To be more precise, a Keras model will be lowered to Relay IR first, then the IR will be transformed as HeteroCL APIs. Then it returns a HeteroCL function that can be executed or used to generate code for other back ends. This PR has two parts. 1. A parser that takes in a Keras model, lowers it into Relay IR and maps to HeteroCL APIs. 2. A library that implements common NN operations.
- Loading branch information
Showing
44 changed files
with
5,474 additions
and
33 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| To properly use this framework, perform the following setup: | ||
| # Using Keras | ||
| Keras uses two different methodologies to build up a neural network: Sequential and Model. Sequential only requires users to insert the different neural layers back to back, while Model requires users to specify exactly how each layer is connected. The differences are shown below: | ||
| ```python | ||
| #sequential Model | ||
| seq_model = Sequential() | ||
| seq_model.add(Dense(32, input_dim=784)) | ||
|
|
||
| #Normal Model | ||
| a = Input(shape=(32,)) | ||
| b = Dense(32)(a) | ||
| mod_model = Model(inputs=a, outputs=b) | ||
|
|
||
| #Save the model | ||
| seq_model.save("my_seq_model.h5") | ||
| mod_model.save("my_model.h5") | ||
|
|
||
| #Load the model | ||
| model = load_model("my_model.h5") | ||
| s_model = load_model("my_seq_model.h5") | ||
|
|
||
| #Executing the model | ||
| func, params = relay_parser.get_relay_model( | ||
| model, (32,), frontend="keras", dtype="float") | ||
| in_x = hcl.asarray(np.random.randint(1, high=10, shape=(32,))) | ||
| out_x = hcl.asarray(np.zeros((32,))) | ||
| func(in_x,*params,out_x) | ||
| print(out_x.asnumpy()) | ||
| ``` | ||
|
|
||
| To insert a model into the HeteroCL framework, you can use *model* directly. If you want to download the model or reload it, perform the code shown in above in the bottom two lines. | ||
| # Using HeteroCL | ||
| 1. Download and setup HeteroCL and TVM from github | ||
| 2. Once both python environments from the githubs are set up, go to the HeteroCL github, and from the main directory, go to the "python/" and "hlib/python/" folders and execute the function "python setup.py install --user" in each. | ||
| Now that the environment is properly set up, here is how to compile a Keras model into a HeteroCL model. | ||
|
|
||
| 1. In a Python script, put into the header "from heterocl.frontend import get_relay_model". | ||
| 2. The function requires the following inputs: (*model*, *shape*, *frontend*, *dtype*, *in_params*.), where *model* is the Keras model, *shape* is the dictionary of inputs, *frontend* is the frontend being used, *dtype* is the data type, and *in_params* is an optional input if the parameters are not included in the model. The function can handle models from two different sources: | ||
| 1. If the model was saved and exported from Keras in an HDF5 file, set ```model``` to the file path to the model. | ||
| 2. If the model is created in the Python script, just set "model" to the Keras model output. | ||
| For the shape inputs, users have to include the inputs name and shape as the key and value to the shape dictionary input. For other inputs like weights that define the model, those parameters do not need to be created by users as the weights are included in the Keras model. The rest of the inputs can be left blank. | ||
| 3. the function outputs a HeteroCL function (```func```) and the list of parameters needed for the model (```params```). To insert an image or tensor into the model, create the input and output tensors by putting in the data as a NumPy array. For inputs, set them as ```in_x = hcl.asarray(numpy_array)``` and for outputs set them as ```out_x = hcl.asarray(np.zeros(out_shape))```. put the inputs and the outputs into their own arrays (eg. ```[in_1,in_2,... in_n]```). | ||
| 4. execute the function as follows: | ||
| ```func(in_array,*params,out_array)```. | ||
| If any of your inputs are a list, prepend an ```*``` to the variable name | ||
| so that way it dumps out all the contents of the list. | ||
| 5. The output is placed into out_array and if you want to convert them back into NumPy use the function ```out_array.asnumpy()```. | ||
|
|
||
| # Setting up ImageNet Dataset | ||
| Since the current ImageNet dataset cannot be download from Keras or | ||
| Tensorflow, if users want to test out models from keras that use | ||
| the Keras Dataset, users will have to setup a numpy file with the script | ||
| ```gen_imagenet.py```. This script allows users to create a numpy array with a given amount of images per class and allows users to set the size of the images to fit their models. | ||
| Before running the script, users have to create a directory called ```imagenet_2012```. Within that directory, create another directory called ```images```. In the ```images``` folder, create two folders called ```train``` and ```val```. The ```gen_imagenet.py``` script along with other scripts out there require this setup. | ||
| From here, obtain the imagenet_2012 zip file and unzip the contents into the proper directory (training images in the ```train``` directory and validation images in the ```val``` directory). From here, run the ```gen_imagenet.py``` script to get data for your model to use. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1,3 @@ | ||
| from . import nn | ||
| from . import op | ||
| from . import frontend | ||
| from . import utils |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| from . import relay_parser | ||
| from . import relay_attr |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,129 @@ | ||
| import hlib | ||
|
|
||
|
|
||
| _convert_map = { | ||
| 'nn.dense': hlib.op.nn.dense, | ||
| 'nn.relu': hlib.op.nn.relu, | ||
| 'nn.bias_add': hlib.op.nn.bias_add, | ||
| # 'prelu' : 'hlib.nn.prelu', | ||
| 'tanh': hlib.op.math.tanh, | ||
| 'sigmoid': hlib.op.math.sigmoid, | ||
| 'nn.softmax': hlib.op.nn.softmax, | ||
| 'nn.leaky_relu': hlib.op.nn.leakyrelu, | ||
| 'exp': hlib.op.math.exp, | ||
| 'log': hlib.op.math.log, | ||
| 'sqrt': hlib.op.math.sqrt, | ||
| 'clip': hlib.op.math.clip, | ||
| 'cast': hlib.op.op.cast, | ||
| 'nn.conv2d': hlib.op.nn.conv2d, | ||
| #'nn.conv2d_transpose': hlib.op.nn.conv2d_transpose, | ||
| 'nn.max_pool2d': hlib.op.nn.max_pool2d, | ||
| 'nn.avg_pool2d': hlib.op.nn.avg_pool2d, | ||
| 'nn.global_max_pool2d': hlib.op.nn.global_max_pool2d, | ||
| 'nn.global_avg_pool2d': hlib.op.nn.global_avg_pool2d, | ||
| 'nn.dropout': hlib.op.nn.dropout, | ||
| 'nn.pad': hlib.op.nn.relay_pad, | ||
| 'transpose': hlib.op.nn.transpose, | ||
| 'reshape': hlib.op.nn.reshape, | ||
| 'nn.batch_flatten': hlib.op.nn.flatten, | ||
| 'nn.batch_norm': hlib.op.nn.batch_norm, | ||
| 'nn.batch_matmul': hlib.op.nn.batch_matmul, | ||
| 'abs': hlib.op.op.abs, | ||
| 'negative': hlib.op.op.negative, | ||
| 'add': hlib.op.op.broadcast_add, | ||
| 'subtract': hlib.op.op.broadcast_sub, | ||
| 'multiply': hlib.op.op.broadcast_mul, | ||
| 'greater': hlib.op.op.broadcast_greater, | ||
| 'divide': hlib.op.op.broadcast_div, | ||
| 'maximum': hlib.op.op.broadcast_max, | ||
| 'concatenate': hlib.op.nn.concatenate, | ||
| 'squeeze': hlib.op.nn.squeeze, | ||
| 'split': hlib.op.nn.split, | ||
| 'full': hlib.op.math.full, | ||
| 'full_like': hlib.op.math.full_like, | ||
| 'zeros': hlib.op.math.zeros, | ||
| 'zeros_like': hlib.op.math.zeros_like, | ||
| 'ones': hlib.op.math.ones, | ||
| 'ones_like': hlib.op.math.ones_like, | ||
| } | ||
|
|
||
|
|
||
| _attrib = { | ||
| 'nn.conv2d': [ | ||
| 'strides', | ||
| 'padding', | ||
| 'dilation', | ||
| 'groups', | ||
| 'channels', | ||
| 'kernel_size', | ||
| 'data_layout', | ||
| 'kernel_layout', | ||
| 'out_layout', | ||
| 'out_dtype'], | ||
| 'nn.conv2d_transpose': [ | ||
| 'channels', | ||
| 'kernel_size', | ||
| 'strides', | ||
| 'padding', | ||
| 'output_padding', | ||
| 'dilation', | ||
| 'groups', | ||
| 'data_layout', | ||
| 'kernel_layout', | ||
| 'out_layout', | ||
| 'out_dtype'], | ||
| 'nn.max_pool2d': [ | ||
| 'pool_size', | ||
| 'strides', | ||
| 'padding', | ||
| 'layout'], | ||
| 'nn.global_max_pool2d': [ | ||
| 'layout'], | ||
| 'nn.global_avg_pool2d': [ | ||
| 'layout'], | ||
| 'nn.dropout': ['rate'], | ||
| 'nn.pad': ['pad_value', 'pad_width'], | ||
| 'nn.avg_pool2d': [ | ||
| 'pool_size', | ||
| 'strides', | ||
| 'padding', | ||
| 'layout'], | ||
| 'transpose': ['axes'], | ||
| 'reshape': [ | ||
| 'newshape'], | ||
| 'squeeze': ['axis'], | ||
| 'cast': ['dtype'], | ||
| 'nn.dense': [ | ||
| 'units', | ||
| 'out_dtype'], | ||
| 'nn.softmax': ['axis'], | ||
| 'nn.bias_add': ['axis'], | ||
| 'sigmoid': [], | ||
| 'tanh': [], | ||
| 'nn.relu': [], | ||
| 'nn.batch_flatten': [], | ||
| 'nn.batch_norm': ['axis', 'epsilon', 'center', 'scale'], | ||
| 'nn.batch_matmul': [], | ||
| 'nn.leaky_relu': ['alpha'], | ||
| 'abs': [], | ||
| 'negative': [], | ||
| 'greater': [], | ||
| 'add': [], | ||
| 'subtract': [], | ||
| 'multiply': [], | ||
| 'divide': [], | ||
| 'maximum': [], | ||
| 'clip': ['a_min', 'a_max'], | ||
| 'concatenate': ['axis'], | ||
| 'squeeze': ['axis'], | ||
| 'split': [ | ||
| 'indices_or_sections', | ||
| 'axis'], | ||
| 'full': ['shape', 'dtype'], | ||
| 'full_like': [], | ||
| 'zeros': ['shape', 'dtype'], | ||
| 'zeros_like': [], | ||
| 'ones_like': [], | ||
| 'exp': [], | ||
| 'log': [] | ||
| } |
Oops, something went wrong.