Skip to content

Commit

Permalink
Merge pull request PaddlePaddle#249 from qili93/add_ipu_api
Browse files Browse the repository at this point in the history
[IPU] add inference python api, test=develop
  • Loading branch information
jiweibo authored May 13, 2022
2 parents 18b95ea + d41625c commit a68454f
Show file tree
Hide file tree
Showing 2 changed files with 47 additions and 0 deletions.
46 changes: 46 additions & 0 deletions docs/api_reference/python_api_doc/Config/IPUConfig.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@

# 使用 IPU 进行预测

API定义如下:

```python
# 启用 IPU 进行预测
# 参数:ipu_device_num - 所需要的IPU个数.
# 参数:ipu_micro_batch_size - 计算图输入的batch size,用于根据输入batch size进行全图Tensor shape推导,仅在动态输入batch size的情况生效
# 参数:ipu_enable_pipelining - 使能IPU间数据流水
# 参数:ipu_batches_per_step - 在使能数据流水的条件下,指定每次跑多少batch的数据,如果关闭数据流水,该值应设置为1
# 返回:None
paddle.inference.Config.enable_ipu(ipu_device_num = 1,
ipu_micro_batch_size = 1,
ipu_enable_pipelining = False,
ipu_batches_per_step = 1)



# 配置 IPU 构图参数
# 参数:ipu_enable_fp16 - 使能float16模式,将float32计算图转换为float16计算图.
# 参数:ipu_replica_num - 设置实例个数,举例ipu_device_num=2,表示单个实例需要2个IPU运行,设置ipu_replica_num=8,表示总共有8个相同实例,所以总共需要16个IPU.
# 参数:ipu_available_memory_proportion - 设置matmul/conv Op可使用的内存比例,取值(0.0, 1.0], 比例越高,计算性能越好.
# 参数:ipu_enable_half_partial - matmul Op中间结果以float16存储于片上.
# 返回:None
paddle.inference.Config.set_ipu_config(ipu_enable_fp16 = False,
ipu_replica_num = 1,
ipu_available_memory_proportion = 1.0,
ipu_enable_half_partial = False)
```

代码示例:

```python
# 引用 paddle inference 预测库
import paddle.inference as paddle_infer

# 创建 config
config = paddle_infer.Config("./mobilenet_v1")

# 启用 IPU,并设置单个实例所需要的IPU个数为1
config.enable_ipu(1)

# 使能float16模式
config.set_ipu_config(True)
```
1 change: 1 addition & 0 deletions docs/api_reference/python_api_doc/Config_index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Config 类
Config/CPUConfig
Config/GPUConfig
Config/XPUConfig
Config/IPUConfig
Config/ORTConfig
Config/OptimConfig
Config/OtherFunction

0 comments on commit a68454f

Please sign in to comment.