-
Notifications
You must be signed in to change notification settings - Fork 762
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update ipu doc #4754
Merged
Merged
Update ipu doc #4754
Changes from 2 commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
9353f90
[IPU] update IPU installatin doc, test=develop
qili93 71e2d4a
Merge branch 'develop' of https://github.com/PaddlePaddle/docs into u…
qili93 49c1c3a
update
qili93 c3adb90
update
qili93 9ac2242
update
qili93 9f6a8c7
update
qili93 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
137 changes: 106 additions & 31 deletions
137
docs/guides/hardware_support/ipu_docs/paddle_install_cn.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,56 +1,131 @@ | ||
# 飞桨框架IPU版安装说明 | ||
|
||
飞桨框架IPU版支持基于IPU的Python的训练和原生推理,目前仅支持通过源代码编译安装。 | ||
飞桨框架IPU版支持基于Graphcore IPU的Python的训练和原生推理,当前支持的Poplar版本为2.5.0, 提供两种安装方式: | ||
|
||
## 通过源代码编译安装 | ||
- Docker镜像方式启动 | ||
- 通过源代码编译安装 | ||
|
||
## Docker镜像方式启动 | ||
|
||
建议在Docker环境内编译和使用飞桨框架IPU版,下面的说明将使用基于Ubuntu18.04的容器进行编译,使用的Python版本为Python3.7。 | ||
当前Docker镜像包含预编译的飞桨框架IPU版,镜像基于Ubuntu18.04基础镜像构建,内置的Python版本为Python3.7。 | ||
|
||
**第一步** 构建Docker镜像 | ||
**第一步** 拉取飞桨框架IPU版镜像 | ||
|
||
```bash | ||
docker pull registry.baidubce.com/device/paddlepaddle:ipu-poplar250 | ||
``` | ||
# 下载源码 | ||
git clone https://github.com/PaddlePaddle/Paddle.git | ||
cd Paddle | ||
|
||
# 构建 Docker 镜像 | ||
docker build -t paddlepaddle/paddle:latest-dev-ipu \ | ||
-f tools/dockerfile/Dockerfile.ipu . | ||
**第二步** 构建并进入Docker容器 | ||
|
||
**注意**:容器启动命令需将主机端的 ipuof 配置文件映射到容器中,可通过设置 IPUOF_CONFIG_PATH 环境变量指向 ipuof 配置文件传入,更多关于 ipuof 配置的信息请访问 [Graphcore: IPUoF configuration file](https://docs.graphcore.ai/projects/vipu-admin/en/latest/cli_reference.html?highlight=ipuof#ipuof-configuration-file)。 | ||
|
||
```bash | ||
export IPUOF_CONFIG_PATH=/opt/ipuof.conf | ||
docker run -it --name liqi-ipu250 -v /home/liqi27:/workspace \ | ||
--shm-size=128G --network=host --ulimit memlock=-1:-1 \ | ||
--cap-add=SYS_PTRACE --security-opt seccomp=unconfined \ | ||
--cap-add=IPC_LOCK --device=/dev/infiniband/ --ipc=host \ | ||
-v ${IPUOF_CONFIG_PATH}:/ipuof.conf -e IPUOF_CONFIG_PATH=/ipuof.conf \ | ||
registry.baidubce.com/device/paddlepaddle:ipu-poplar250 /bin/bash | ||
``` | ||
|
||
**第三步** 检查容器运行环境 | ||
|
||
```bash | ||
# 检查容器是否可以正确识别海光DCU设备 | ||
gc-monitor | ||
# 预期得到如下结果 | ||
+---------------+--------------------------------------------------------------------------------+ | ||
| gc-monitor | Partition: ipuof [active] has 4 reconfigurable IPUs | | ||
+-------------+--------------------+--------+--------------+----------+------+----+------+-------+ | ||
| IPU-M | Serial |IPU-M SW|Server version| ICU FW | Type | ID | IPU# |Routing| | ||
+-------------+--------------------+--------+--------------+----------+------+----+------+-------+ | ||
|...31.100.130| 0134.0002.8210321 | | 1.8.1 | 2.3.5 |M2000 | 0 | 3 | DNC | | ||
|...31.100.130| 0134.0002.8210321 | | 1.8.1 | 2.3.5 |M2000 | 1 | 2 | DNC | | ||
|...31.100.130| 0134.0001.8210321 | | 1.8.1 | 2.3.5 |M2000 | 2 | 1 | DNC | | ||
|...31.100.130| 0134.0001.8210321 | | 1.8.1 | 2.3.5 |M2000 | 3 | 0 | DNC | | ||
+-------------+--------------------+--------+--------------+----------+------+----+------+-------+ | ||
+--------------------------------------------------------------------------------------------------+ | ||
| No attached processes in partition ipuof | | ||
+--------------------------------------------------------------------------------------------------+ | ||
|
||
# 检查飞桨框架IPU版已经安装 | ||
pip list | grep paddlepaddle-ipu | ||
# 预期得到如下结果 | ||
paddlepaddle-ipu 0.0.0.dev250 | ||
|
||
# 检查飞桨框架IPU版正常工作 | ||
python -c "import paddle; paddle.utils.run_check()" | ||
# 预期得到如下结果 | ||
Running verify PaddlePaddle program ... | ||
PaddlePaddle works well on 1 CPU. | ||
PaddlePaddle works well on 2 CPUs. | ||
PaddlePaddle is installed successfully! Let's start deep learning with PaddlePaddle now. | ||
``` | ||
|
||
**第二步** 下载Paddle源码并编译 | ||
## 通过源代码编译安装 | ||
|
||
**预先要求**:建议在Docker环境内进行飞桨框架IPU版的源码编译,容器环境配置和启动命令请参考上一章节的内容。 | ||
|
||
**第一步**:检查容器编译环境 | ||
|
||
请在编译之前,检查如下的环境变量是否正确,如果没有则需要安装相应的依赖库,并导出相应的环境变量。 | ||
|
||
```bash | ||
# PATH 中存在 GCC/G++ 8.2 | ||
export PATH=/opt/compiler/gcc-8.2/bin:${PATH} | ||
|
||
# PATH 中存在 cmake 3.16.0 | ||
export PATH=/opt/cmake-3.16/bin:${PATH} | ||
|
||
# PATH 与 LD_LIBRARY_PATH 中存在 popart 与 poplar | ||
export PATH=/opt/popart/bin:/opt/poplar/lib:${PATH} | ||
export LD_LIBRARY_PATH=/opt/popart/lib:/opt/poplar/lib:${LD_LIBRARY_PATH} | ||
|
||
# PATH 中存在 Python 3.7 | ||
# 注意:镜像中的 python 3.7 通过 miniconda 安装,请通过 conda activate base 命令加载Python 3.7环境 | ||
export PATH=/opt/conda/bin:${PATH} | ||
``` | ||
# 创建并运行 Docker 容器 | ||
# 需要将主机端的 ipuof 配置文件映射到容器中,可通过设置 IPUOF_CONFIG_PATH 环境变量指向 ipuof 配置文件传入 | ||
# 更多关于 ipuof 配置的信息可访问 https://docs.graphcore.ai/projects/vipu-admin/en/latest/cli_reference.html?highlight=ipuof#ipuof-configuration-file | ||
docker run --ulimit memlock=-1:-1 --net=host --cap-add=IPC_LOCK \ | ||
--device=/dev/infiniband/ --ipc=host \ | ||
--name paddle-dev-ipu -w /home \ | ||
-v ${IPUOF_CONFIG_PATH}:/ipuof.conf \ | ||
-e IPUOF_CONFIG_PATH=/ipuof.conf \ | ||
-it paddlepaddle/paddle:latest-dev-ipu bash | ||
|
||
# 容器内下载源码 | ||
|
||
**第二步**:下载Paddle源码并编译,CMAKE编译选项含义请参见[编译选项表](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/install/Tables.html#Compile) | ||
|
||
```bash | ||
# 下载源码,默认 develop 分支 | ||
git clone https://github.com/PaddlePaddle/Paddle.git | ||
cd Paddle | ||
|
||
# 创建编译目录 | ||
mkdir build && cd build | ||
|
||
# 执行 CMake | ||
cmake .. -DWITH_IPU=ON -DWITH_PYTHON=ON -DPY_VERSION=3.7 -DWITH_MKL=ON \ | ||
-DPOPLAR_DIR=/opt/poplar -DPOPART_DIR=/opt/popart -DCMAKE_BUILD_TYPE=Release | ||
# 执行cmake | ||
cmake .. -DPY_VERSION=3.7 -DWITH_IPU=ON -DWITH_MKL=ON \ | ||
-DPOPLAR_DIR=/opt/poplar -DPOPART_DIR=/opt/popart \ | ||
-DWITH_TESTING=ON -DCMAKE_BUILD_TYPE=Release | ||
|
||
# 开始编译 | ||
# 使用以下命令来编译 | ||
make -j$(nproc) | ||
|
||
# 安装编译生成的 wheel 包 | ||
pip install -U python/dist/paddlepaddle-0.0.0-cp37-cp37m-linux_x86_64.whl | ||
``` | ||
|
||
**第三步** 验证安装 | ||
**第三步**:安装与验证编译生成的wheel包 | ||
|
||
``` | ||
编译完成之后进入`Paddle/build/python/dist`目录即可找到编译生成的.whl安装包,安装与验证命令如下: | ||
|
||
```bash | ||
# 安装命令 | ||
python -m pip install -U paddlepaddle_ipu-0.0.0-cp37-cp37m-linux_x86_64.whl | ||
|
||
# 验证命令 | ||
python -c "import paddle; paddle.utils.run_check()" | ||
# 预期得到如下结果 | ||
Running verify PaddlePaddle program ... | ||
PaddlePaddle works well on 1 CPU. | ||
PaddlePaddle works well on 2 CPUs. | ||
PaddlePaddle is installed successfully! Let's start deep learning with PaddlePaddle now. | ||
``` | ||
## 如何卸载 | ||
|
||
请使用以下命令卸载Paddle: | ||
|
||
```bash | ||
pip uninstall paddlepaddle-ipu | ||
``` |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
->
检查容器是否可以正确识别IPU设备
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed, thanks