Skip to content

Commit

Permalink
[CMSIS-NN] Update microNPU demo to include offloading to CMSIS-NN (#9979
Browse files Browse the repository at this point in the history
)

* [CMSIS-NN] Update microNPU demo to include offloading to CMSIS-NN

Change-Id: I6a3ba9db3e3cb2bd7c10383ebd52f9a1cdad74d0

* [CMSIS-NN] Update microNPU demo to include offloading to CMSIS-NN

* Addressing comments

Change-Id: I98fcdf95bf408700968827e1abd084a916b3b21c

* [CMSIS-NN] Update microNPU demo to include offloading to CMSIS-NN

    * Addressing comments
    * Remove build folder before running demo to address #10020

Change-Id: Ifa7ad3ff431f427f8afb8b3c9f06711b3b59ad62

* Correctly filter tvmc Targets

Fixed logic to check for >2 TVM Target to be based on none-hybrid
Targets only

Co-authored-by: Chris Sidebottom <[email protected]>
  • Loading branch information
grant-arm and Mousius authored Jan 24, 2022
1 parent d066441 commit 65b4b09
Show file tree
Hide file tree
Showing 7 changed files with 52 additions and 25 deletions.
13 changes: 12 additions & 1 deletion apps/microtvm/ethosu/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -40,11 +40,16 @@ PKG_CFLAGS = ${PKG_COMPILE_OPTS} \
-I${ETHOSU_PATH}/core_driver/include \
-I${CMSIS_PATH}/Device/ARM/${ARM_CPU}/Include/ \
-I${CMSIS_PATH}/CMSIS/Core/Include \
-I${CMSIS_PATH}/CMSIS/NN/Include \
-I${CMSIS_PATH}/CMSIS/DSP/Include \
-I$(abspath $(BUILD_DIR))/codegen/host/include \
-DETHOSU_TEST_RUNNER_TOL=${ETHOSU_TEST_RUNNER_TOL}
DRIVER_CMAKE_FLAGS = -DCMAKE_TOOLCHAIN_FILE=$(abspath $(BUILD_DIR))/../arm-none-eabi-gcc.cmake \
-DETHOSU_LOG_SEVERITY=debug \
-DCMAKE_SYSTEM_PROCESSOR=cortex-m55
CMSIS_NN_CMAKE_FLAGS = -DCMAKE_TOOLCHAIN_FILE=$(abspath $(BUILD_DIR))/../arm-none-eabi-gcc.cmake \
-DTARGET_CPU=cortex-m55 \
-DBUILD_CMSIS_NN_FUNCTIONS=YES
PKG_LDFLAGS = -lm -specs=nosys.specs -static -T corstone300.ld

$(ifeq VERBOSE,1)
Expand Down Expand Up @@ -87,8 +92,14 @@ ${BUILD_DIR}/ethosu_core_driver/libethosu_core_driver.a:
$(QUIET)cd $(ETHOSU_DRIVER_PATH) && $(CMAKE) -B $(abspath $(BUILD_DIR)/ethosu_core_driver) $(DRIVER_CMAKE_FLAGS)
$(QUIET)cd $(abspath $(BUILD_DIR)/ethosu_core_driver) && $(MAKE)

# Build CMSIS-NN Softmax
${BUILD_DIR}/cmsis_nn/Source/SoftmaxFunctions/libCMSISNNSoftmax.a:
$(QUIET)mkdir -p $(@D)
$(QUIET)cd $(CMSIS_PATH)/CMSIS/NN && $(CMAKE) -B $(abspath $(BUILD_DIR)/cmsis_nn) $(CMSIS_NN_CMAKE_FLAGS)
$(QUIET)cd $(abspath $(BUILD_DIR)/cmsis_nn) && $(MAKE) CMSISNNSoftmax

# Build demo application
$(BUILD_DIR)/demo: src/demo.c src/tvm_ethosu_runtime.c $(UART_SRCS) $(BUILD_DIR)/stack_allocator.o $(BUILD_DIR)/crt_backend_api.o ${BUILD_DIR}/libcodegen.a ${BUILD_DIR}/libcmsis_startup.a ${BUILD_DIR}/ethosu_core_driver/libethosu_core_driver.a
$(BUILD_DIR)/demo: src/demo.c src/tvm_ethosu_runtime.c $(UART_SRCS) $(BUILD_DIR)/stack_allocator.o $(BUILD_DIR)/crt_backend_api.o ${BUILD_DIR}/libcodegen.a ${BUILD_DIR}/libcmsis_startup.a ${BUILD_DIR}/ethosu_core_driver/libethosu_core_driver.a ${BUILD_DIR}/cmsis_nn/Source/SoftmaxFunctions/libCMSISNNSoftmax.a
$(QUIET)mkdir -p $(@D)
$(QUIET)$(CC) $(PKG_CFLAGS) -o $@ $^ $(PKG_LDFLAGS)

Expand Down
19 changes: 10 additions & 9 deletions apps/microtvm/ethosu/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,11 @@
<!--- under the License. -->


Running TVM on bare metal Arm(R) Cortex(R)-M55 CPU and Ethos(TM)-U55 NPU
========================================================================
Running TVM on bare metal Arm(R) Cortex(R)-M55 CPU, Ethos(TM)-U55 NPU and CMSIS-NN
==================================================================================

This folder contains an example of how to use TVM to run a model
on bare metal Cortex(R)-M55 CPU and Ethos(TM)-U55 NPU.
on bare metal Cortex(R)-M55 CPU, Ethos(TM)-U55 NPU and CMSIS-NN.

Prerequisites
-------------
Expand All @@ -45,6 +45,7 @@ You will also need TVM which can either be:
- Built from source (see [Install from Source](https://tvm.apache.org/docs/install/from_source.html))
- When building from source, the following need to be set in config.cmake:
- set(USE_ETHOSU ON)
- set(USE_CMSISNN ON)
- set(USE_MICRO ON)
- set(USE_LLVM ON)
- Installed from TLCPack(see [TLCPack](https://tlcpack.ai/))
Expand Down Expand Up @@ -72,15 +73,15 @@ the locations for these can be specified as arguments to run_demo.sh, for exampl
```
This will:
- Download a quantized mobilenet v1 model
- Use tvmc to compile the model for Cortex(R)-M55 CPU and Ethos(TM)-U55 NPU
- Download an image of a kitten to run the model on
- Download a quantized (int8) mobilenet v2 model
- Use tvmc to compile the model for Cortex(R)-M55 CPU, Ethos(TM)-U55 NPU and CMSIS-NN
- Download an image of a penguin to run the model on
- Create a C header file inputs.c containing the image data as a C array
- Create a C header file outputs.c containing a C array where the output of inference will be stored
- Build the Ethos(TM)-U55 core driver
- Build the demo application
- Run the demo application on a Fixed Virtual Platform (FVP) based on Arm(R) Corstone(TM)-300 software
- The application will display what the image has been classified as e.g. "The image has been classified as 'tabby'"
- The application will display what the image has been classified as e.g. "The image has been classified as 'king penguin'"
Using your own image
--------------------
Expand All @@ -90,6 +91,6 @@ image to be converted into an array of bytes for consumption by the model.
The demo can be modified to use an image of your choice by changing the following lines in run_demo.sh
```bash
curl -sS https://s3.amazonaws.com/model-server/inputs/kitten.jpg -o ./kitten.jpg
python3 ./convert_image.py ./kitten.jpg
curl -sS https://upload.wikimedia.org/wikipedia/commons/1/18/Falkland_Islands_Penguins_29.jpg -o penguin.jpg
python3 ./convert_image.py ./build/penguin.jpg
```
11 changes: 6 additions & 5 deletions apps/microtvm/ethosu/convert_image.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,9 @@ def create_header_file(name, section, tensor_name, tensor_data, output_path):
header_file.write(
"#include <tvmgen_default.h>\n"
+ f"const size_t {tensor_name}_len = {tensor_data.size};\n"
+ f'uint8_t {tensor_name}[] __attribute__((section("{section}"), aligned(16))) = "'
+ f'int8_t {tensor_name}[] __attribute__((section("{section}"), aligned(16))) = "'
)

data_hexstr = tensor_data.tobytes().hex()
for i in range(0, len(data_hexstr), 2):
header_file.write(f"\\x{data_hexstr[i:i+2]}")
Expand All @@ -52,14 +53,14 @@ def create_headers(image_name):
resized_image = Image.open(img_path).resize((224, 224))
img_data = np.asarray(resized_image).astype("float32")

# Convert input to NCHW
img_data = np.transpose(img_data, (2, 0, 1))
# # Add the batch dimension, as we are expecting 4-dimensional input: NCHW.
img_data = np.expand_dims(img_data, axis=0)

# Create input header file
input_data = img_data.astype(np.uint8)
input_data = img_data.astype(np.int8)
create_header_file("inputs", "ethosu_scratch", "input", input_data, "./include")
# Create output header file
output_data = np.zeros([1001], np.uint8)
output_data = np.zeros([1001], np.int8)
create_header_file(
"outputs",
"output_data_sec",
Expand Down
15 changes: 8 additions & 7 deletions apps/microtvm/ethosu/run_demo.sh
Original file line number Diff line number Diff line change
Expand Up @@ -120,35 +120,36 @@ done
script_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"

# Make build directory
make cleanall
mkdir -p build
cd build

# Get mobilenet_v1 tflite model
mobilenet_url='https://storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224_quant.tgz'
curl --retry 64 -sSL ${mobilenet_url} | gunzip | tar -xvf - ./mobilenet_v1_1.0_224_quant.tflite
# Get mobilenet_v2 tflite model
mobilenet_url='https://github.com/ARM-software/ML-zoo/raw/b9e26e662c00e0c0b23587888e75ac1205a99b6e/models/image_classification/mobilenet_v2_1.0_224/tflite_int8/mobilenet_v2_1.0_224_INT8.tflite'
curl --retry 64 -sSL ${mobilenet_url} -o ./mobilenet_v2_1.0_224_INT8.tflite

# Compile model for Arm(R) Cortex(R)-M55 CPU and Ethos(TM)-U55 NPU
# An alternative to using "python3 -m tvm.driver.tvmc" is to call
# "tvmc" directly once TVM has been pip installed.
python3 -m tvm.driver.tvmc compile --target="ethos-u -accelerator_config=ethos-u55-256, c" \
python3 -m tvm.driver.tvmc compile --target="ethos-u -accelerator_config=ethos-u55-256, cmsis-nn, c" \
--target-c-mcpu=cortex-m55 \
--runtime=crt \
--executor=aot \
--executor-aot-interface-api=c \
--executor-aot-unpacked-api=1 \
--pass-config tir.disable_vectorize=1 ./mobilenet_v1_1.0_224_quant.tflite --output-format=mlf
--pass-config tir.disable_vectorize=1 ./mobilenet_v2_1.0_224_INT8.tflite --output-format=mlf
tar -xvf module.tar

# Get ImageNet labels
curl -sS https://raw.githubusercontent.com/tensorflow/tensorflow/master/tensorflow/lite/java/demo/app/src/main/assets/labels_mobilenet_quant_v1_224.txt \
-o ./labels_mobilenet_quant_v1_224.txt

# Get input image
curl -sS https://s3.amazonaws.com/model-server/inputs/kitten.jpg -o ./kitten.jpg
curl -sS https://upload.wikimedia.org/wikipedia/commons/1/18/Falkland_Islands_Penguins_29.jpg -o penguin.jpg

# Create C header files
cd ..
python3 ./convert_image.py ./build/kitten.jpg
python3 ./convert_image.py ./build/penguin.jpg
python3 ./convert_labels.py ./build/labels_mobilenet_quant_v1_224.txt

# Build demo executable
Expand Down
4 changes: 2 additions & 2 deletions apps/microtvm/ethosu/src/demo.c
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ int main(int argc, char** argv) {
.output = output,
};
struct tvmgen_default_inputs inputs = {
.input = input,
.tfl_quantize = input,
};
struct ethosu_driver* driver = ethosu_reserve_driver();
struct tvmgen_default_devices devices = {
Expand All @@ -53,7 +53,7 @@ int main(int argc, char** argv) {
ethosu_release_driver(driver);

// Calculate index of max value
uint8_t max_value = 0;
int8_t max_value = -128;
int32_t max_index = -1;
for (unsigned int i = 0; i < output_len; ++i) {
if (output[i] > max_value) {
Expand Down
2 changes: 1 addition & 1 deletion python/tvm/driver/tvmc/target.py
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ def validate_targets(parse_targets, additional_target_options=None):
f"The last target needs to be a TVM target. Choices: {tvm_target_names}"
)

tvm_targets = [t for t in targets if t in tvm_target_kinds]
tvm_targets = [t for t in targets if t in _valid_target_kinds()]
if len(tvm_targets) > 2:
verbose_tvm_targets = ", ".join(tvm_targets)
raise TVMCException(
Expand Down
13 changes: 13 additions & 0 deletions tests/python/driver/tvmc/test_target.py
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,19 @@ def test_parse_hybrid_target():
assert targets[1]["is_tvm_target"]


def test_parse_multiple_hybrid_target():
"""Hybrid Target and multiple external codegen"""
targets = parse_target("ethos-u,cmsis-nn,c")

assert len(targets) == 3
assert "ethos-u" == targets[0]["name"]
assert not targets[0]["is_tvm_target"]
assert "cmsis-nn" == targets[1]["name"]
assert not targets[1]["is_tvm_target"]
assert "c" == targets[2]["name"]
assert targets[2]["is_tvm_target"]


def test_parse_quotes_and_separators_on_options():
targets_no_quote = parse_target("foo -option1=+v1.0x,+value,+bar")
targets_single_quote = parse_target("foo -option1='+v1.0x,+value'")
Expand Down

0 comments on commit 65b4b09

Please sign in to comment.