Support different dtypes of inputs for broadcast for dropout optimization #52093
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
你的PR提交成功，感谢你对开源项目的贡献! |
zhangbopd
force-pushed
the
dropout_grad_opt_brcast
branch
from
833dcee
to
9971126
Compare
zhangbopd
force-pushed
the
dropout_grad_opt_brcast
branch
from
9b7e5bd
to
df71250
Compare
zhangbopd
changed the title
zhangbopd
requested review from
AnnaTrainingG,
JamesLim-sy,
Xreki,
zhangting2020,
sneaxiy and
ZzSean
JamesLim-sy
reviewed
Apr 12, 2023
Shixiaowei02
previously approved these changes
Apr 13, 2023
|
BTW, 请补充针对Dropout_grad 优化前后的性能数据 |
OK，这部分工作在 #52969 中清理 ElementwiseType 和 InT 时完成，性能数据放在对应PR描述中 |
|
Sorry to inform you that a6119cc's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually. |
JamesLim-sy
reviewed
Apr 25, 2023
shaojiewang
reviewed
Apr 27, 2023
zhangbopd
dismissed stale reviews from ZzSean, Shixiaowei02, and jiweibo
via
a5c47c5
luotao1
approved these changes
Apr 27, 2023
jiweibo
approved these changes
Apr 27, 2023
zhangbopd
added a commit
to zhangbopd/Paddle
that referenced
this pull request
May 9, 2023
…tion (PaddlePaddle#52093) * change judgement for DropoutGradGPUKernelDriver * add UnrollerWithoutVecSize and after this Loaddata to be refined * pass unittest * use same unroller with XPU * BroadcastWithInt64Index * BroadcastDataLoader template partial specialization * fix compile errs in ROCms * PR comment
XiaoguangHu01
pushed a commit
that referenced
this pull request
May 10, 2023
…to Release/2.5 (#53623) * Support different dtypes of inputs for broadcast for dropout optimization (#52093) * change judgement for DropoutGradGPUKernelDriver * add UnrollerWithoutVecSize and after this Loaddata to be refined * pass unittest * use same unroller with XPU * BroadcastWithInt64Index * BroadcastDataLoader template partial specialization * fix compile errs in ROCms * PR comment * dropout_nd_optimization (#51479) * with printf * add DropOutNdForwardKernel * PR comment * Dropout optimize & clean broadcast inT and ElementwiseType (#52969) * change judgement for DropoutGradGPUKernelDriver * add UnrollerWithoutVecSize and after this Loaddata to be refined * pass unittest * use same unroller with XPU * BroadcastWithInt64Index * BroadcastDataLoader template partial specialization * fix compile errs in ROCms * clean ElementwiseT and InT for BroadcastKernel * default axis and clean inT * remove redundant fast divmod computation * optimize drop_nd & drop_nd_grad * optimize BroadcastDataLoader bf16 fp16 * rm InT etc. after merge develop * delete constexpr for windows ci * fix conflict * fix conflic with develop * fix conflic * new clean * clean * Fix xpu2 kp compile error (#53548) * fix conflict * conflict
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
Performance optimization
PR changes
OPs
Description
This PR aims to support different dtypes of inputs for Broadcast so as to do
dropoutGraddropout_ndanddropout_ndGradoptimization (dropout optimization with the same method was done previously)InTBroadcastDataLoadertemplate partial specialization andBroadcastLoadTypefor high performance in some casesBroadcastWithInt64Index