From c8bb315cb22f588a6e8c1b9ff6103eff85f50378 Mon Sep 17 00:00:00 2001 From: Yuwen Hu Date: Thu, 15 Sep 2022 14:37:19 +0800 Subject: [PATCH 1/3] Update Trainer to InferenOptimizer for related how-to guides --- .../accelerate_pytorch_inference_onnx.ipynb | 14 ++++++------ ...ccelerate_pytorch_inference_openvino.ipynb | 12 +++++----- .../quantize_pytorch_inference_inc.ipynb | 22 +++++++++---------- .../quantize_pytorch_inference_pot.ipynb | 16 +++++++------- 4 files changed, 32 insertions(+), 32 deletions(-) diff --git a/python/nano/tutorial/notebook/inference/pytorch/accelerate_pytorch_inference_onnx.ipynb b/python/nano/tutorial/notebook/inference/pytorch/accelerate_pytorch_inference_onnx.ipynb index 9f39c132aa2..294e40fcd01 100644 --- a/python/nano/tutorial/notebook/inference/pytorch/accelerate_pytorch_inference_onnx.ipynb +++ b/python/nano/tutorial/notebook/inference/pytorch/accelerate_pytorch_inference_onnx.ipynb @@ -32,7 +32,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "You can use ``Trainer.trace(..., accelerator='onnxruntime')`` API to enable the ONNXRuntime acceleration for PyTorch inference. It only takes a few lines." + "You can use ``InferenceOptimizer.trace(..., accelerator='onnxruntime')`` API to enable the ONNXRuntime acceleration for PyTorch inference. It only takes a few lines." ] }, { @@ -109,7 +109,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "To enable ONNXRuntime acceleration for your PyTorch inference pipeline, **the only change you need to made is to import BigDL-Nano Trainer, and trace your PyTorch model to convert it into an ONNXRuntime accelerated module for inference**:" + "To enable ONNXRuntime acceleration for your PyTorch inference pipeline, **the only change you need to made is to import BigDL-Nano** `InferenceOptimizer`**, and trace your PyTorch model to convert it into an ONNXRuntime accelerated module for inference**:" ] }, { @@ -119,11 +119,11 @@ "outputs": [], "source": [ "import torch\n", - "from bigdl.nano.pytorch import Trainer\n", + "from bigdl.nano.pytorch import InferenceOptimizer\n", "\n", - "ort_model = Trainer.trace(model_ft,\n", - " accelerator=\"onnxruntime\",\n", - " input_sample=torch.rand(1, 3, 224, 224))" + "ort_model = InferenceOptimizer.trace(model_ft,\n", + " accelerator=\"onnxruntime\",\n", + " input_sample=torch.rand(1, 3, 224, 224))" ] }, { @@ -134,7 +134,7 @@ "> \n", "> `input_sample` is the parameter for ONNXRuntime accelerator to know the **shape** of the model input. So both the batch size and the specific values are not important to `input_sample`. If we want our test dataset to consist of images with $224 \\times 224$ pixels, we could use `torch.rand(1, 3, 224, 224)` for `input_sample` here. \n", "> \n", - "> Please refer to [API documentation](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.Trainer.trace) for more information on `Trainer.trace`." + "> Please refer to [API documentation](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.InferenceOptimizer.trace) for more information on `InferenceOptimizer.trace`." ] }, { diff --git a/python/nano/tutorial/notebook/inference/pytorch/accelerate_pytorch_inference_openvino.ipynb b/python/nano/tutorial/notebook/inference/pytorch/accelerate_pytorch_inference_openvino.ipynb index c5460527101..c66f13a0622 100644 --- a/python/nano/tutorial/notebook/inference/pytorch/accelerate_pytorch_inference_openvino.ipynb +++ b/python/nano/tutorial/notebook/inference/pytorch/accelerate_pytorch_inference_openvino.ipynb @@ -32,7 +32,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "You can use `Trainer.trace(..., accelerator='openvino')` API to enable the OpenVINO acceleration for PyTorch inference. It only takes a few lines." + "You can use `InferenceOptimizer.trace(..., accelerator='openvino')` API to enable the OpenVINO acceleration for PyTorch inference. It only takes a few lines." ] }, { @@ -79,7 +79,7 @@ "source": [ "> ⚠️ **Warning**\n", ">\n", - "> Errors may occur when using `Trainer.trace(accelerator='openvino')` API in CentOS, becuase the latest version of `openvino-dev` is not supported in CentOS." + "> Errors may occur when using `InferenceOptimizer.trace(..., accelerator='openvino')` API in CentOS, becuase the latest version of `openvino-dev` is not supported in CentOS." ] }, { @@ -120,7 +120,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "To enable OpenVINO acceleration for your PyTorch inference pipeline, **the only change you need to made is to import BigDL-Nano Trainer, and trace your PyTorch model to convert it into an OpenVINO accelerated module for inference**:" + "To enable OpenVINO acceleration for your PyTorch inference pipeline, **the only change you need to made is to import BigDL-Nano** `InferenceOptimizer`**, and trace your PyTorch model to convert it into an OpenVINO accelerated module for inference**:" ] }, { @@ -130,9 +130,9 @@ "outputs": [], "source": [ "import torch\n", - "from bigdl.nano.pytorch import Trainer\n", + "from bigdl.nano.pytorch import InferenceOptimizer\n", "\n", - "ov_model = Trainer.trace(model_ft,\n", + "ov_model = InferenceOptimizer.trace(model_ft,\n", " accelerator=\"openvino\",\n", " input_sample=torch.rand(1, 3, 224, 224))" ] @@ -145,7 +145,7 @@ "> \n", "> `input_sample` is the parameter for OpenVINO accelerator to know the **shape** of the model input. So both the batch size and the specific values are not important to `input_sample`. If we want our test dataset to consist of images with $224 \\times 224$ pixels, we could use `torch.rand(1, 3, 224, 224)` for `input_sample` here.\n", "> \n", - "> Please refer to [API documentation](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.Trainer.trace) for more information on `Trainer.trace`." + "> Please refer to [API documentation](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.InferenceOptimizer.trace) for more information on `InferenceOptimizer.trace`." ] }, { diff --git a/python/nano/tutorial/notebook/inference/pytorch/quantize_pytorch_inference_inc.ipynb b/python/nano/tutorial/notebook/inference/pytorch/quantize_pytorch_inference_inc.ipynb index dcc94cfddea..3955fcfdb0b 100644 --- a/python/nano/tutorial/notebook/inference/pytorch/quantize_pytorch_inference_inc.ipynb +++ b/python/nano/tutorial/notebook/inference/pytorch/quantize_pytorch_inference_inc.ipynb @@ -32,7 +32,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "With Intel Neural Compressor (INC) as quantization engine, you can apply `Trainer.quantize` API to realize post-training quantization on your PyTorch `nn.Module`. `Trainer.quantize` also supports ONNXRuntime acceleration at the meantime through specifying `accelerator='onnxruntime'`. All acceleration takes only a few lines." + "With Intel Neural Compressor (INC) as quantization engine, you can apply `InferenceOptimizer.quantize` API to realize post-training quantization on your PyTorch `nn.Module`. `InferenceOptimizer.quantize` also supports ONNXRuntime acceleration at the meantime through specifying `accelerator='onnxruntime'`. All acceleration takes only a few lines." ] }, { @@ -201,7 +201,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "To enable quantization using INC for inference, you could simply **import BigDL-Nano Trainer, and use Trainer to quantize your PyTorch model**:" + "To enable quantization using INC for inference, you could simply **import BigDL-Nano** `InferenceOptimizer`**, and use** `InferenceOptimizer` **to quantize your PyTorch model**:" ] }, { @@ -210,10 +210,10 @@ "metadata": {}, "outputs": [], "source": [ - "from bigdl.nano.pytorch import Trainer\n", + "from bigdl.nano.pytorch import InferenceOptimizer\n", "\n", - "q_model = Trainer.quantize(model, \n", - " calib_dataloader=DataLoader(train_dataset, batch_size=32))" + "q_model = InferenceOptimizer.quantize(model, \n", + " calib_dataloader=DataLoader(train_dataset, batch_size=32))" ] }, { @@ -229,11 +229,11 @@ "metadata": {}, "outputs": [], "source": [ - "from bigdl.nano.pytorch import Trainer\n", + "from bigdl.nano.pytorch import InferenceOptimizer\n", "\n", - "q_model = Trainer.quantize(model,\n", - " accelerator='onnxruntime',\n", - " calib_dataloader=DataLoader(train_dataset, batch_size=32))" + "q_model = InferenceOptimizer.quantize(model,\n", + " accelerator='onnxruntime',\n", + " calib_dataloader=DataLoader(train_dataset, batch_size=32))" ] }, { @@ -242,11 +242,11 @@ "source": [ "> 📝 **Note**\n", "> \n", - "> `Trainer` will by default quantize your PyTorch `nn.Module` through **static** post-training quantization. For this case, `calib_dataloader` (for calibration data) is required. Batch size is not important to ``calib_dataloader``, as it intends to read 100 samples. And there could be no label in calibration data.\n", + "> `InferenceOptimizer` will by default quantize your PyTorch `nn.Module` through **static** post-training quantization. For this case, `calib_dataloader` (for calibration data) is required. Batch size is not important to ``calib_dataloader``, as it intends to read 100 samples. And there could be no label in calibration data.\n", "> \n", "> If you would like to implement dynamic post-training quantization, you could set parameter `approach='dynamic'`. In this case, `calib_dataloader` should be `None`. Compared to dynamic quantization, static quantization could lead to faster inference as it eliminates the data conversion costs between layers.\n", "> \n", - "> Please refer to [API documentation](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.Trainer.quantize) for more information on `Trainer.quantize`." + "> Please refer to [API documentation](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.InferenceOptimizer.quantize) for more information on `InferenceOptimizer.quantize`." ] }, { diff --git a/python/nano/tutorial/notebook/inference/pytorch/quantize_pytorch_inference_pot.ipynb b/python/nano/tutorial/notebook/inference/pytorch/quantize_pytorch_inference_pot.ipynb index c011c39b84b..0ab1a4c4a7a 100644 --- a/python/nano/tutorial/notebook/inference/pytorch/quantize_pytorch_inference_pot.ipynb +++ b/python/nano/tutorial/notebook/inference/pytorch/quantize_pytorch_inference_pot.ipynb @@ -32,7 +32,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "As Post-training Optimization Tools (POT) is provided by OpenVINO toolkit, OpenVINO acceleration will be enabled in the meantime when using POT for quantization. You can call `Trainer.quantize` API with `accelerator='openvino'` to use POT for your PyTorch `nn.Module`. It only takes a few lines." + "As Post-training Optimization Tools (POT) is provided by OpenVINO toolkit, OpenVINO acceleration will be enabled in the meantime when using POT for quantization. You can call `InferenceOptimizer.quantize` API with `accelerator='openvino'` to use POT for your PyTorch `nn.Module`. It only takes a few lines." ] }, { @@ -79,7 +79,7 @@ "source": [ "> ⚠️ **Warning**\n", ">\n", - "> Errors may occur when using `Trainer.quantize(..., accelerator='openvino')` API in CentOS, becuase the latest version of `openvino-dev` is not supported in CentOS." + "> Errors may occur when using `InferenceOptimizer.quantize(..., accelerator='openvino')` API in CentOS, becuase the latest version of `openvino-dev` is not supported in CentOS." ] }, { @@ -193,7 +193,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "To enable quantization using POT for inference, you could simply **import BigDL-Nano Trainer, and use Trainer to quantize your PyTorch model**:" + "To enable quantization using POT for inference, you could simply **import BigDL-Nano** `InferenceOptimizer`**, and use** `InferenceOptimizer` **to quantize your PyTorch model**:" ] }, { @@ -202,11 +202,11 @@ "metadata": {}, "outputs": [], "source": [ - "from bigdl.nano.pytorch import Trainer\n", + "from bigdl.nano.pytorch import InferenceOptimizer\n", "\n", - "q_model = Trainer.quantize(model,\n", - " accelerator='openvino',\n", - " calib_dataloader=DataLoader(train_dataset, batch_size=32))" + "q_model = InferenceOptimizer.quantize(model,\n", + " accelerator='openvino',\n", + " calib_dataloader=DataLoader(train_dataset, batch_size=32))" ] }, { @@ -219,7 +219,7 @@ "> \n", "> For `calib_dataloader`, batch size is not important as it intends to read 100 samples. And there could be no label in calibration data.\n", "> \n", - "> Please refer to [API documentation](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.Trainer.quantize) for more information on `Trainer.quantize`." + "> Please refer to [API documentation](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.InferenceOptimizer.quantize) for more information on `InferenceOptimizer.quantize`." ] }, { From 3ee5ea50988574b50eb79dab8a1a110b63e6970f Mon Sep 17 00:00:00 2001 From: Yuwen Hu Date: Thu, 15 Sep 2022 14:39:19 +0800 Subject: [PATCH 2/3] Disable other nano tests temporarily --- .github/workflows/nano_notebooks_tests.yml | 5 +++-- .github/workflows/nano_unit_tests_basic.yml | 5 +++-- .github/workflows/nano_unit_tests_pytorch.yml | 5 +++-- .github/workflows/nano_unit_tests_tensorflow.yml | 5 +++-- 4 files changed, 12 insertions(+), 8 deletions(-) diff --git a/.github/workflows/nano_notebooks_tests.yml b/.github/workflows/nano_notebooks_tests.yml index a0e1587a022..7237f3cf5e3 100644 --- a/.github/workflows/nano_notebooks_tests.yml +++ b/.github/workflows/nano_notebooks_tests.yml @@ -8,8 +8,9 @@ on: pull_request: branches: [ main ] paths: - - 'python/nano/**' - - '.github/workflows/nano_notebooks_tests.yml' + #- 'python/nano/**' + #- '.github/workflows/nano_notebooks_tests.yml' + - 'apps/**' # A workflow run is made up of one or more jobs that can run sequentially or in parallel diff --git a/.github/workflows/nano_unit_tests_basic.yml b/.github/workflows/nano_unit_tests_basic.yml index a1edf526b13..7acff03f768 100644 --- a/.github/workflows/nano_unit_tests_basic.yml +++ b/.github/workflows/nano_unit_tests_basic.yml @@ -8,8 +8,9 @@ on: pull_request: branches: [ main ] paths: - - 'python/nano/**' - - '.github/workflows/nano_unit_tests_basic.yml' + #- 'python/nano/**' + #- '.github/workflows/nano_unit_tests_basic.yml' + - 'apps/**' # A workflow run is made up of one or more jobs that can run sequentially or in parallel jobs: diff --git a/.github/workflows/nano_unit_tests_pytorch.yml b/.github/workflows/nano_unit_tests_pytorch.yml index 799bc81f7ea..e43d38eea6c 100644 --- a/.github/workflows/nano_unit_tests_pytorch.yml +++ b/.github/workflows/nano_unit_tests_pytorch.yml @@ -8,8 +8,9 @@ on: pull_request: branches: [ main ] paths: - - 'python/nano/**' - - '.github/workflows/nano_unit_tests_pytorch.yml' + #- 'python/nano/**' + #- '.github/workflows/nano_unit_tests_pytorch.yml' + - 'apps/**' # A workflow run is made up of one or more jobs that can run sequentially or in parallel jobs: diff --git a/.github/workflows/nano_unit_tests_tensorflow.yml b/.github/workflows/nano_unit_tests_tensorflow.yml index b420ad5e8f6..6d46f22d565 100644 --- a/.github/workflows/nano_unit_tests_tensorflow.yml +++ b/.github/workflows/nano_unit_tests_tensorflow.yml @@ -8,8 +8,9 @@ on: pull_request: branches: [ main ] paths: - - 'python/nano/**' - - '.github/workflows/nano_unit_tests_tensorflow.yml' + #- 'python/nano/**' + #- '.github/workflows/nano_unit_tests_tensorflow.yml' + - 'apps/**' # A workflow run is made up of one or more jobs that can run sequentially or in parallel jobs: From 140552bada68a6691933ab3b71c89980fa8fd9be Mon Sep 17 00:00:00 2001 From: Yuwen Hu Date: Thu, 15 Sep 2022 16:09:15 +0800 Subject: [PATCH 3/3] Enable other nano tests again --- .github/workflows/nano_notebooks_tests.yml | 5 ++--- .github/workflows/nano_unit_tests_basic.yml | 5 ++--- .github/workflows/nano_unit_tests_pytorch.yml | 5 ++--- .github/workflows/nano_unit_tests_tensorflow.yml | 5 ++--- 4 files changed, 8 insertions(+), 12 deletions(-) diff --git a/.github/workflows/nano_notebooks_tests.yml b/.github/workflows/nano_notebooks_tests.yml index 7237f3cf5e3..a0e1587a022 100644 --- a/.github/workflows/nano_notebooks_tests.yml +++ b/.github/workflows/nano_notebooks_tests.yml @@ -8,9 +8,8 @@ on: pull_request: branches: [ main ] paths: - #- 'python/nano/**' - #- '.github/workflows/nano_notebooks_tests.yml' - - 'apps/**' + - 'python/nano/**' + - '.github/workflows/nano_notebooks_tests.yml' # A workflow run is made up of one or more jobs that can run sequentially or in parallel diff --git a/.github/workflows/nano_unit_tests_basic.yml b/.github/workflows/nano_unit_tests_basic.yml index 7acff03f768..a1edf526b13 100644 --- a/.github/workflows/nano_unit_tests_basic.yml +++ b/.github/workflows/nano_unit_tests_basic.yml @@ -8,9 +8,8 @@ on: pull_request: branches: [ main ] paths: - #- 'python/nano/**' - #- '.github/workflows/nano_unit_tests_basic.yml' - - 'apps/**' + - 'python/nano/**' + - '.github/workflows/nano_unit_tests_basic.yml' # A workflow run is made up of one or more jobs that can run sequentially or in parallel jobs: diff --git a/.github/workflows/nano_unit_tests_pytorch.yml b/.github/workflows/nano_unit_tests_pytorch.yml index e43d38eea6c..799bc81f7ea 100644 --- a/.github/workflows/nano_unit_tests_pytorch.yml +++ b/.github/workflows/nano_unit_tests_pytorch.yml @@ -8,9 +8,8 @@ on: pull_request: branches: [ main ] paths: - #- 'python/nano/**' - #- '.github/workflows/nano_unit_tests_pytorch.yml' - - 'apps/**' + - 'python/nano/**' + - '.github/workflows/nano_unit_tests_pytorch.yml' # A workflow run is made up of one or more jobs that can run sequentially or in parallel jobs: diff --git a/.github/workflows/nano_unit_tests_tensorflow.yml b/.github/workflows/nano_unit_tests_tensorflow.yml index 6d46f22d565..b420ad5e8f6 100644 --- a/.github/workflows/nano_unit_tests_tensorflow.yml +++ b/.github/workflows/nano_unit_tests_tensorflow.yml @@ -8,9 +8,8 @@ on: pull_request: branches: [ main ] paths: - #- 'python/nano/**' - #- '.github/workflows/nano_unit_tests_tensorflow.yml' - - 'apps/**' + - 'python/nano/**' + - '.github/workflows/nano_unit_tests_tensorflow.yml' # A workflow run is made up of one or more jobs that can run sequentially or in parallel jobs: