Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update ipu doc #4754

Merged
merged 6 commits into from
May 13, 2022
Merged
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
137 changes: 106 additions & 31 deletions docs/guides/hardware_support/ipu_docs/paddle_install_cn.md
Original file line number Diff line number Diff line change
@@ -1,56 +1,131 @@
# 飞桨框架IPU版安装说明

飞桨框架IPU版支持基于IPU的Python的训练和原生推理,目前仅支持通过源代码编译安装。
飞桨框架IPU版支持基于Graphcore IPU的Python的训练和原生推理,当前支持的Poplar版本为2.5.0, 提供两种安装方式:

## 通过源代码编译安装
- Docker镜像方式启动
- 通过源代码编译安装

## Docker镜像方式启动

建议在Docker环境内编译和使用飞桨框架IPU版,下面的说明将使用基于Ubuntu18.04的容器进行编译,使用的Python版本为Python3.7。
当前Docker镜像包含预编译的飞桨框架IPU版,镜像基于Ubuntu18.04基础镜像构建,内置的Python版本为Python3.7。

**第一步** 构建Docker镜像
**第一步** 拉取飞桨框架IPU版镜像

```bash
docker pull registry.baidubce.com/device/paddlepaddle:ipu-poplar250
```
# 下载源码
git clone https://github.com/PaddlePaddle/Paddle.git
cd Paddle

# 构建 Docker 镜像
docker build -t paddlepaddle/paddle:latest-dev-ipu \
-f tools/dockerfile/Dockerfile.ipu .
**第二步** 构建并进入Docker容器

**注意**:容器启动命令需将主机端的 ipuof 配置文件映射到容器中,可通过设置 IPUOF_CONFIG_PATH 环境变量指向 ipuof 配置文件传入,更多关于 ipuof 配置的信息请访问 [Graphcore: IPUoF configuration file](https://docs.graphcore.ai/projects/vipu-admin/en/latest/cli_reference.html?highlight=ipuof#ipuof-configuration-file)。

```bash
export IPUOF_CONFIG_PATH=/opt/ipuof.conf
docker run -it --name liqi-ipu250 -v /home/liqi27:/workspace \
--shm-size=128G --network=host --ulimit memlock=-1:-1 \
--cap-add=SYS_PTRACE --security-opt seccomp=unconfined \
--cap-add=IPC_LOCK --device=/dev/infiniband/ --ipc=host \
-v ${IPUOF_CONFIG_PATH}:/ipuof.conf -e IPUOF_CONFIG_PATH=/ipuof.conf \
registry.baidubce.com/device/paddlepaddle:ipu-poplar250 /bin/bash
```

**第三步** 检查容器运行环境

```bash
# 检查容器是否可以正确识别海光DCU设备
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

->

检查容器是否可以正确识别IPU设备

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, thanks

gc-monitor
# 预期得到如下结果
+---------------+--------------------------------------------------------------------------------+
| gc-monitor | Partition: ipuof [active] has 4 reconfigurable IPUs |
+-------------+--------------------+--------+--------------+----------+------+----+------+-------+
| IPU-M | Serial |IPU-M SW|Server version| ICU FW | Type | ID | IPU# |Routing|
+-------------+--------------------+--------+--------------+----------+------+----+------+-------+
|...31.100.130| 0134.0002.8210321 | | 1.8.1 | 2.3.5 |M2000 | 0 | 3 | DNC |
|...31.100.130| 0134.0002.8210321 | | 1.8.1 | 2.3.5 |M2000 | 1 | 2 | DNC |
|...31.100.130| 0134.0001.8210321 | | 1.8.1 | 2.3.5 |M2000 | 2 | 1 | DNC |
|...31.100.130| 0134.0001.8210321 | | 1.8.1 | 2.3.5 |M2000 | 3 | 0 | DNC |
+-------------+--------------------+--------+--------------+----------+------+----+------+-------+
+--------------------------------------------------------------------------------------------------+
| No attached processes in partition ipuof |
+--------------------------------------------------------------------------------------------------+

# 检查飞桨框架IPU版已经安装
pip list | grep paddlepaddle-ipu
# 预期得到如下结果
paddlepaddle-ipu 0.0.0.dev250

# 检查飞桨框架IPU版正常工作
python -c "import paddle; paddle.utils.run_check()"
# 预期得到如下结果
Running verify PaddlePaddle program ...
PaddlePaddle works well on 1 CPU.
PaddlePaddle works well on 2 CPUs.
PaddlePaddle is installed successfully! Let's start deep learning with PaddlePaddle now.
```

**第二步** 下载Paddle源码并编译
## 通过源代码编译安装

**预先要求**:建议在Docker环境内进行飞桨框架IPU版的源码编译,容器环境配置和启动命令请参考上一章节的内容。

**第一步**:检查容器编译环境

请在编译之前,检查如下的环境变量是否正确,如果没有则需要安装相应的依赖库,并导出相应的环境变量。

```bash
# PATH 中存在 GCC/G++ 8.2
export PATH=/opt/compiler/gcc-8.2/bin:${PATH}

# PATH 中存在 cmake 3.16.0
export PATH=/opt/cmake-3.16/bin:${PATH}

# PATH 与 LD_LIBRARY_PATH 中存在 popart 与 poplar
export PATH=/opt/popart/bin:/opt/poplar/lib:${PATH}
export LD_LIBRARY_PATH=/opt/popart/lib:/opt/poplar/lib:${LD_LIBRARY_PATH}

# PATH 中存在 Python 3.7
# 注意:镜像中的 python 3.7 通过 miniconda 安装,请通过 conda activate base 命令加载Python 3.7环境
export PATH=/opt/conda/bin:${PATH}
```
# 创建并运行 Docker 容器
# 需要将主机端的 ipuof 配置文件映射到容器中,可通过设置 IPUOF_CONFIG_PATH 环境变量指向 ipuof 配置文件传入
# 更多关于 ipuof 配置的信息可访问 https://docs.graphcore.ai/projects/vipu-admin/en/latest/cli_reference.html?highlight=ipuof#ipuof-configuration-file
docker run --ulimit memlock=-1:-1 --net=host --cap-add=IPC_LOCK \
--device=/dev/infiniband/ --ipc=host \
--name paddle-dev-ipu -w /home \
-v ${IPUOF_CONFIG_PATH}:/ipuof.conf \
-e IPUOF_CONFIG_PATH=/ipuof.conf \
-it paddlepaddle/paddle:latest-dev-ipu bash

# 容器内下载源码

**第二步**:下载Paddle源码并编译,CMAKE编译选项含义请参见[编译选项表](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/install/Tables.html#Compile)

```bash
# 下载源码,默认 develop 分支
git clone https://github.com/PaddlePaddle/Paddle.git
cd Paddle

# 创建编译目录
mkdir build && cd build

# 执行 CMake
cmake .. -DWITH_IPU=ON -DWITH_PYTHON=ON -DPY_VERSION=3.7 -DWITH_MKL=ON \
-DPOPLAR_DIR=/opt/poplar -DPOPART_DIR=/opt/popart -DCMAKE_BUILD_TYPE=Release
# 执行cmake
cmake .. -DPY_VERSION=3.7 -DWITH_IPU=ON -DWITH_MKL=ON \
-DPOPLAR_DIR=/opt/poplar -DPOPART_DIR=/opt/popart \
-DWITH_TESTING=ON -DCMAKE_BUILD_TYPE=Release

# 开始编译
# 使用以下命令来编译
make -j$(nproc)

# 安装编译生成的 wheel 包
pip install -U python/dist/paddlepaddle-0.0.0-cp37-cp37m-linux_x86_64.whl
```

**第三步** 验证安装
**第三步**:安装与验证编译生成的wheel包

```
编译完成之后进入`Paddle/build/python/dist`目录即可找到编译生成的.whl安装包,安装与验证命令如下:

```bash
# 安装命令
python -m pip install -U paddlepaddle_ipu-0.0.0-cp37-cp37m-linux_x86_64.whl

# 验证命令
python -c "import paddle; paddle.utils.run_check()"
# 预期得到如下结果
Running verify PaddlePaddle program ...
PaddlePaddle works well on 1 CPU.
PaddlePaddle works well on 2 CPUs.
PaddlePaddle is installed successfully! Let's start deep learning with PaddlePaddle now.
```
## 如何卸载

请使用以下命令卸载Paddle:

```bash
pip uninstall paddlepaddle-ipu
```