Skip to content

Commit

Permalink
[Nano] Update Trainer -> InferenceOptimizer for Related How-to Guides (…
Browse files Browse the repository at this point in the history 
…#5778)

* Update Trainer to InferenOptimizer for related how-to guides

* Disable other nano tests temporarily

* Enable other nano tests again
  • Loading branch information
@Oscilloscope98
Oscilloscope98 authored Sep 15, 2022
1 parent 679a544 commit 7a12fbe
Show file tree
Hide file tree
Showing 4 changed files with 32 additions and 32 deletions.
14 changes: 7 additions & 7 deletions python/nano/tutorial/notebook/inference/pytorch/accelerate_pytorch_inference_onnx.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"You can use ``Trainer.trace(..., accelerator='onnxruntime')`` API to enable the ONNXRuntime acceleration for PyTorch inference. It only takes a few lines."
"You can use ``InferenceOptimizer.trace(..., accelerator='onnxruntime')`` API to enable the ONNXRuntime acceleration for PyTorch inference. It only takes a few lines."
]
},
{
Expand Down Expand Up @@ -109,7 +109,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"To enable ONNXRuntime acceleration for your PyTorch inference pipeline, **the only change you need to made is to import BigDL-Nano Trainer, and trace your PyTorch model to convert it into an ONNXRuntime accelerated module for inference**:"
"To enable ONNXRuntime acceleration for your PyTorch inference pipeline, **the only change you need to made is to import BigDL-Nano** `InferenceOptimizer`**, and trace your PyTorch model to convert it into an ONNXRuntime accelerated module for inference**:"
]
},
{
Expand All @@ -119,11 +119,11 @@
"outputs": [],
"source": [
"import torch\n",
"from bigdl.nano.pytorch import Trainer\n",
"from bigdl.nano.pytorch import InferenceOptimizer\n",
"\n",
"ort_model = Trainer.trace(model_ft,\n",
" accelerator=\"onnxruntime\",\n",
" input_sample=torch.rand(1, 3, 224, 224))"
"ort_model = InferenceOptimizer.trace(model_ft,\n",
" accelerator=\"onnxruntime\",\n",
" input_sample=torch.rand(1, 3, 224, 224))"
]
},
{
Expand All @@ -134,7 +134,7 @@
"> \n",
"> `input_sample` is the parameter for ONNXRuntime accelerator to know the **shape** of the model input. So both the batch size and the specific values are not important to `input_sample`. If we want our test dataset to consist of images with $224 \\times 224$ pixels, we could use `torch.rand(1, 3, 224, 224)` for `input_sample` here. \n",
"> \n",
"> Please refer to [API documentation](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.Trainer.trace) for more information on `Trainer.trace`."
"> Please refer to [API documentation](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.InferenceOptimizer.trace) for more information on `InferenceOptimizer.trace`."
]
},
{
Expand Down
12 changes: 6 additions & 6 deletions python/nano/tutorial/notebook/inference/pytorch/accelerate_pytorch_inference_openvino.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"You can use `Trainer.trace(..., accelerator='openvino')` API to enable the OpenVINO acceleration for PyTorch inference. It only takes a few lines."
"You can use `InferenceOptimizer.trace(..., accelerator='openvino')` API to enable the OpenVINO acceleration for PyTorch inference. It only takes a few lines."
]
},
{
Expand Down Expand Up @@ -79,7 +79,7 @@
"source": [
"> ⚠️ **Warning**\n",
">\n",
"> Errors may occur when using `Trainer.trace(accelerator='openvino')` API in CentOS, becuase the latest version of `openvino-dev` is not supported in CentOS."
"> Errors may occur when using `InferenceOptimizer.trace(..., accelerator='openvino')` API in CentOS, becuase the latest version of `openvino-dev` is not supported in CentOS."
]
},
{
Expand Down Expand Up @@ -120,7 +120,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"To enable OpenVINO acceleration for your PyTorch inference pipeline, **the only change you need to made is to import BigDL-Nano Trainer, and trace your PyTorch model to convert it into an OpenVINO accelerated module for inference**:"
"To enable OpenVINO acceleration for your PyTorch inference pipeline, **the only change you need to made is to import BigDL-Nano** `InferenceOptimizer`**, and trace your PyTorch model to convert it into an OpenVINO accelerated module for inference**:"
]
},
{
Expand All @@ -130,9 +130,9 @@
"outputs": [],
"source": [
"import torch\n",
"from bigdl.nano.pytorch import Trainer\n",
"from bigdl.nano.pytorch import InferenceOptimizer\n",
"\n",
"ov_model = Trainer.trace(model_ft,\n",
"ov_model = InferenceOptimizer.trace(model_ft,\n",
" accelerator=\"openvino\",\n",
" input_sample=torch.rand(1, 3, 224, 224))"
]
Expand All @@ -145,7 +145,7 @@
"> \n",
"> `input_sample` is the parameter for OpenVINO accelerator to know the **shape** of the model input. So both the batch size and the specific values are not important to `input_sample`. If we want our test dataset to consist of images with $224 \\times 224$ pixels, we could use `torch.rand(1, 3, 224, 224)` for `input_sample` here.\n",
"> \n",
"> Please refer to [API documentation](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.Trainer.trace) for more information on `Trainer.trace`."
"> Please refer to [API documentation](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.InferenceOptimizer.trace) for more information on `InferenceOptimizer.trace`."
]
},
{
Expand Down
22 changes: 11 additions & 11 deletions python/nano/tutorial/notebook/inference/pytorch/quantize_pytorch_inference_inc.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"With Intel Neural Compressor (INC) as quantization engine, you can apply `Trainer.quantize` API to realize post-training quantization on your PyTorch `nn.Module`. `Trainer.quantize` also supports ONNXRuntime acceleration at the meantime through specifying `accelerator='onnxruntime'`. All acceleration takes only a few lines."
"With Intel Neural Compressor (INC) as quantization engine, you can apply `InferenceOptimizer.quantize` API to realize post-training quantization on your PyTorch `nn.Module`. `InferenceOptimizer.quantize` also supports ONNXRuntime acceleration at the meantime through specifying `accelerator='onnxruntime'`. All acceleration takes only a few lines."
]
},
{
Expand Down Expand Up @@ -201,7 +201,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"To enable quantization using INC for inference, you could simply **import BigDL-Nano Trainer, and use Trainer to quantize your PyTorch model**:"
"To enable quantization using INC for inference, you could simply **import BigDL-Nano** `InferenceOptimizer`**, and use** `InferenceOptimizer` **to quantize your PyTorch model**:"
]
},
{
Expand All @@ -210,10 +210,10 @@
"metadata": {},
"outputs": [],
"source": [
"from bigdl.nano.pytorch import Trainer\n",
"from bigdl.nano.pytorch import InferenceOptimizer\n",
"\n",
"q_model = Trainer.quantize(model, \n",
" calib_dataloader=DataLoader(train_dataset, batch_size=32))"
"q_model = InferenceOptimizer.quantize(model, \n",
" calib_dataloader=DataLoader(train_dataset, batch_size=32))"
]
},
{
Expand All @@ -229,11 +229,11 @@
"metadata": {},
"outputs": [],
"source": [
"from bigdl.nano.pytorch import Trainer\n",
"from bigdl.nano.pytorch import InferenceOptimizer\n",
"\n",
"q_model = Trainer.quantize(model,\n",
" accelerator='onnxruntime',\n",
" calib_dataloader=DataLoader(train_dataset, batch_size=32))"
"q_model = InferenceOptimizer.quantize(model,\n",
" accelerator='onnxruntime',\n",
" calib_dataloader=DataLoader(train_dataset, batch_size=32))"
]
},
{
Expand All @@ -242,11 +242,11 @@
"source": [
"> 📝 **Note**\n",
"> \n",
"> `Trainer` will by default quantize your PyTorch `nn.Module` through **static** post-training quantization. For this case, `calib_dataloader` (for calibration data) is required. Batch size is not important to ``calib_dataloader``, as it intends to read 100 samples. And there could be no label in calibration data.\n",
"> `InferenceOptimizer` will by default quantize your PyTorch `nn.Module` through **static** post-training quantization. For this case, `calib_dataloader` (for calibration data) is required. Batch size is not important to ``calib_dataloader``, as it intends to read 100 samples. And there could be no label in calibration data.\n",
"> \n",
"> If you would like to implement dynamic post-training quantization, you could set parameter `approach='dynamic'`. In this case, `calib_dataloader` should be `None`. Compared to dynamic quantization, static quantization could lead to faster inference as it eliminates the data conversion costs between layers.\n",
"> \n",
"> Please refer to [API documentation](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.Trainer.quantize) for more information on `Trainer.quantize`."
"> Please refer to [API documentation](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.InferenceOptimizer.quantize) for more information on `InferenceOptimizer.quantize`."
]
},
{
Expand Down
16 changes: 8 additions & 8 deletions python/nano/tutorial/notebook/inference/pytorch/quantize_pytorch_inference_pot.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"As Post-training Optimization Tools (POT) is provided by OpenVINO toolkit, OpenVINO acceleration will be enabled in the meantime when using POT for quantization. You can call `Trainer.quantize` API with `accelerator='openvino'` to use POT for your PyTorch `nn.Module`. It only takes a few lines."
"As Post-training Optimization Tools (POT) is provided by OpenVINO toolkit, OpenVINO acceleration will be enabled in the meantime when using POT for quantization. You can call `InferenceOptimizer.quantize` API with `accelerator='openvino'` to use POT for your PyTorch `nn.Module`. It only takes a few lines."
]
},
{
Expand Down Expand Up @@ -79,7 +79,7 @@
"source": [
"> ⚠️ **Warning**\n",
">\n",
"> Errors may occur when using `Trainer.quantize(..., accelerator='openvino')` API in CentOS, becuase the latest version of `openvino-dev` is not supported in CentOS."
"> Errors may occur when using `InferenceOptimizer.quantize(..., accelerator='openvino')` API in CentOS, becuase the latest version of `openvino-dev` is not supported in CentOS."
]
},
{
Expand Down Expand Up @@ -193,7 +193,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"To enable quantization using POT for inference, you could simply **import BigDL-Nano Trainer, and use Trainer to quantize your PyTorch model**:"
"To enable quantization using POT for inference, you could simply **import BigDL-Nano** `InferenceOptimizer`**, and use** `InferenceOptimizer` **to quantize your PyTorch model**:"
]
},
{
Expand All @@ -202,11 +202,11 @@
"metadata": {},
"outputs": [],
"source": [
"from bigdl.nano.pytorch import Trainer\n",
"from bigdl.nano.pytorch import InferenceOptimizer\n",
"\n",
"q_model = Trainer.quantize(model,\n",
" accelerator='openvino',\n",
" calib_dataloader=DataLoader(train_dataset, batch_size=32))"
"q_model = InferenceOptimizer.quantize(model,\n",
" accelerator='openvino',\n",
" calib_dataloader=DataLoader(train_dataset, batch_size=32))"
]
},
{
Expand All @@ -219,7 +219,7 @@
"> \n",
"> For `calib_dataloader`, batch size is not important as it intends to read 100 samples. And there could be no label in calibration data.\n",
"> \n",
"> Please refer to [API documentation](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.Trainer.quantize) for more information on `Trainer.quantize`."
"> Please refer to [API documentation](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.InferenceOptimizer.quantize) for more information on `InferenceOptimizer.quantize`."
]
},
{
Expand Down

0 comments on commit 7a12fbe

Please sign in to comment.