Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Nano] Update Trainer -> InferenceOptimizer for Related How-to Guides #5778

Merged
merged 3 commits into from
Sep 15, 2022
Merged

[Nano] Update Trainer -> InferenceOptimizer for Related How-to Guides #5778

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c8bb315
Update Trainer to InferenOptimizer for related how-to guides
Oscilloscope98 Sep 15, 2022
3ee5ea5
Disable other nano tests temporarily
Oscilloscope98 Sep 15, 2022
140552b
Enable other nano tests again
Oscilloscope98 Sep 15, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions python/nano/tutorial/notebook/inference/pytorch/accelerate_pytorch_inference_onnx.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"You can use ``Trainer.trace(..., accelerator='onnxruntime')`` API to enable the ONNXRuntime acceleration for PyTorch inference. It only takes a few lines."
"You can use ``InferenceOptimizer.trace(..., accelerator='onnxruntime')`` API to enable the ONNXRuntime acceleration for PyTorch inference. It only takes a few lines."
]
},
{
Expand Down Expand Up @@ -109,7 +109,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"To enable ONNXRuntime acceleration for your PyTorch inference pipeline, **the only change you need to made is to import BigDL-Nano Trainer, and trace your PyTorch model to convert it into an ONNXRuntime accelerated module for inference**:"
"To enable ONNXRuntime acceleration for your PyTorch inference pipeline, **the only change you need to made is to import BigDL-Nano** `InferenceOptimizer`**, and trace your PyTorch model to convert it into an ONNXRuntime accelerated module for inference**:"
]
},
{
Expand All @@ -119,11 +119,11 @@
"outputs": [],
"source": [
"import torch\n",
"from bigdl.nano.pytorch import Trainer\n",
"from bigdl.nano.pytorch import InferenceOptimizer\n",
"\n",
"ort_model = Trainer.trace(model_ft,\n",
" accelerator=\"onnxruntime\",\n",
" input_sample=torch.rand(1, 3, 224, 224))"
"ort_model = InferenceOptimizer.trace(model_ft,\n",
" accelerator=\"onnxruntime\",\n",
" input_sample=torch.rand(1, 3, 224, 224))"
]
},
{
Expand All @@ -134,7 +134,7 @@
"> \n",
"> `input_sample` is the parameter for ONNXRuntime accelerator to know the **shape** of the model input. So both the batch size and the specific values are not important to `input_sample`. If we want our test dataset to consist of images with $224 \\times 224$ pixels, we could use `torch.rand(1, 3, 224, 224)` for `input_sample` here. \n",
"> \n",
"> Please refer to [API documentation](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.Trainer.trace) for more information on `Trainer.trace`."
"> Please refer to [API documentation](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.InferenceOptimizer.trace) for more information on `InferenceOptimizer.trace`."
]
},
{
Expand Down
12 changes: 6 additions & 6 deletions python/nano/tutorial/notebook/inference/pytorch/accelerate_pytorch_inference_openvino.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"You can use `Trainer.trace(..., accelerator='openvino')` API to enable the OpenVINO acceleration for PyTorch inference. It only takes a few lines."
"You can use `InferenceOptimizer.trace(..., accelerator='openvino')` API to enable the OpenVINO acceleration for PyTorch inference. It only takes a few lines."
]
},
{
Expand Down Expand Up @@ -79,7 +79,7 @@
"source": [
"> ⚠️ **Warning**\n",
">\n",
"> Errors may occur when using `Trainer.trace(accelerator='openvino')` API in CentOS, becuase the latest version of `openvino-dev` is not supported in CentOS."
"> Errors may occur when using `InferenceOptimizer.trace(..., accelerator='openvino')` API in CentOS, becuase the latest version of `openvino-dev` is not supported in CentOS."
]
},
{
Expand Down Expand Up @@ -120,7 +120,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"To enable OpenVINO acceleration for your PyTorch inference pipeline, **the only change you need to made is to import BigDL-Nano Trainer, and trace your PyTorch model to convert it into an OpenVINO accelerated module for inference**:"
"To enable OpenVINO acceleration for your PyTorch inference pipeline, **the only change you need to made is to import BigDL-Nano** `InferenceOptimizer`**, and trace your PyTorch model to convert it into an OpenVINO accelerated module for inference**:"
]
},
{
Expand All @@ -130,9 +130,9 @@
"outputs": [],
"source": [
"import torch\n",
"from bigdl.nano.pytorch import Trainer\n",
"from bigdl.nano.pytorch import InferenceOptimizer\n",
"\n",
"ov_model = Trainer.trace(model_ft,\n",
"ov_model = InferenceOptimizer.trace(model_ft,\n",
" accelerator=\"openvino\",\n",
" input_sample=torch.rand(1, 3, 224, 224))"
]
Expand All @@ -145,7 +145,7 @@
"> \n",
"> `input_sample` is the parameter for OpenVINO accelerator to know the **shape** of the model input. So both the batch size and the specific values are not important to `input_sample`. If we want our test dataset to consist of images with $224 \\times 224$ pixels, we could use `torch.rand(1, 3, 224, 224)` for `input_sample` here.\n",
"> \n",
"> Please refer to [API documentation](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.Trainer.trace) for more information on `Trainer.trace`."
"> Please refer to [API documentation](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.InferenceOptimizer.trace) for more information on `InferenceOptimizer.trace`."
]
},
{
Expand Down
22 changes: 11 additions & 11 deletions python/nano/tutorial/notebook/inference/pytorch/quantize_pytorch_inference_inc.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"With Intel Neural Compressor (INC) as quantization engine, you can apply `Trainer.quantize` API to realize post-training quantization on your PyTorch `nn.Module`. `Trainer.quantize` also supports ONNXRuntime acceleration at the meantime through specifying `accelerator='onnxruntime'`. All acceleration takes only a few lines."
"With Intel Neural Compressor (INC) as quantization engine, you can apply `InferenceOptimizer.quantize` API to realize post-training quantization on your PyTorch `nn.Module`. `InferenceOptimizer.quantize` also supports ONNXRuntime acceleration at the meantime through specifying `accelerator='onnxruntime'`. All acceleration takes only a few lines."
]
},
{
Expand Down Expand Up @@ -201,7 +201,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"To enable quantization using INC for inference, you could simply **import BigDL-Nano Trainer, and use Trainer to quantize your PyTorch model**:"
"To enable quantization using INC for inference, you could simply **import BigDL-Nano** `InferenceOptimizer`**, and use** `InferenceOptimizer` **to quantize your PyTorch model**:"
]
},
{
Expand All @@ -210,10 +210,10 @@
"metadata": {},
"outputs": [],
"source": [
"from bigdl.nano.pytorch import Trainer\n",
"from bigdl.nano.pytorch import InferenceOptimizer\n",
"\n",
"q_model = Trainer.quantize(model, \n",
" calib_dataloader=DataLoader(train_dataset, batch_size=32))"
"q_model = InferenceOptimizer.quantize(model, \n",
" calib_dataloader=DataLoader(train_dataset, batch_size=32))"
]
},
{
Expand All @@ -229,11 +229,11 @@
"metadata": {},
"outputs": [],
"source": [
"from bigdl.nano.pytorch import Trainer\n",
"from bigdl.nano.pytorch import InferenceOptimizer\n",
"\n",
"q_model = Trainer.quantize(model,\n",
" accelerator='onnxruntime',\n",
" calib_dataloader=DataLoader(train_dataset, batch_size=32))"
"q_model = InferenceOptimizer.quantize(model,\n",
" accelerator='onnxruntime',\n",
" calib_dataloader=DataLoader(train_dataset, batch_size=32))"
]
},
{
Expand All @@ -242,11 +242,11 @@
"source": [
"> 📝 **Note**\n",
"> \n",
"> `Trainer` will by default quantize your PyTorch `nn.Module` through **static** post-training quantization. For this case, `calib_dataloader` (for calibration data) is required. Batch size is not important to ``calib_dataloader``, as it intends to read 100 samples. And there could be no label in calibration data.\n",
"> `InferenceOptimizer` will by default quantize your PyTorch `nn.Module` through **static** post-training quantization. For this case, `calib_dataloader` (for calibration data) is required. Batch size is not important to ``calib_dataloader``, as it intends to read 100 samples. And there could be no label in calibration data.\n",
"> \n",
"> If you would like to implement dynamic post-training quantization, you could set parameter `approach='dynamic'`. In this case, `calib_dataloader` should be `None`. Compared to dynamic quantization, static quantization could lead to faster inference as it eliminates the data conversion costs between layers.\n",
"> \n",
"> Please refer to [API documentation](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.Trainer.quantize) for more information on `Trainer.quantize`."
"> Please refer to [API documentation](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.InferenceOptimizer.quantize) for more information on `InferenceOptimizer.quantize`."
]
},
{
Expand Down
16 changes: 8 additions & 8 deletions python/nano/tutorial/notebook/inference/pytorch/quantize_pytorch_inference_pot.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"As Post-training Optimization Tools (POT) is provided by OpenVINO toolkit, OpenVINO acceleration will be enabled in the meantime when using POT for quantization. You can call `Trainer.quantize` API with `accelerator='openvino'` to use POT for your PyTorch `nn.Module`. It only takes a few lines."
"As Post-training Optimization Tools (POT) is provided by OpenVINO toolkit, OpenVINO acceleration will be enabled in the meantime when using POT for quantization. You can call `InferenceOptimizer.quantize` API with `accelerator='openvino'` to use POT for your PyTorch `nn.Module`. It only takes a few lines."
]
},
{
Expand Down Expand Up @@ -79,7 +79,7 @@
"source": [
"> ⚠️ **Warning**\n",
">\n",
"> Errors may occur when using `Trainer.quantize(..., accelerator='openvino')` API in CentOS, becuase the latest version of `openvino-dev` is not supported in CentOS."
"> Errors may occur when using `InferenceOptimizer.quantize(..., accelerator='openvino')` API in CentOS, becuase the latest version of `openvino-dev` is not supported in CentOS."
]
},
{
Expand Down Expand Up @@ -193,7 +193,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"To enable quantization using POT for inference, you could simply **import BigDL-Nano Trainer, and use Trainer to quantize your PyTorch model**:"
"To enable quantization using POT for inference, you could simply **import BigDL-Nano** `InferenceOptimizer`**, and use** `InferenceOptimizer` **to quantize your PyTorch model**:"
]
},
{
Expand All @@ -202,11 +202,11 @@
"metadata": {},
"outputs": [],
"source": [
"from bigdl.nano.pytorch import Trainer\n",
"from bigdl.nano.pytorch import InferenceOptimizer\n",
"\n",
"q_model = Trainer.quantize(model,\n",
" accelerator='openvino',\n",
" calib_dataloader=DataLoader(train_dataset, batch_size=32))"
"q_model = InferenceOptimizer.quantize(model,\n",
" accelerator='openvino',\n",
" calib_dataloader=DataLoader(train_dataset, batch_size=32))"
]
},
{
Expand All @@ -219,7 +219,7 @@
"> \n",
"> For `calib_dataloader`, batch size is not important as it intends to read 100 samples. And there could be no label in calibration data.\n",
"> \n",
"> Please refer to [API documentation](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.Trainer.quantize) for more information on `Trainer.quantize`."
"> Please refer to [API documentation](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.InferenceOptimizer.quantize) for more information on `InferenceOptimizer.quantize`."
]
},
{
Expand Down