modify clip #5080
Merged
modify clip #5080
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
感谢你贡献飞桨文档，文档预览构建中，Docs-New 跑完后即可预览，预览链接：http://preview-pr-5080.paddle-docs-preview.paddlepaddle.org.cn/documentation/docs/zh/api/index_cn.html |
|
✅ This PR's description meets the template requirements! |
momozi1996
reviewed
Jul 14, 2022
| @@ -6,7 +6,7 @@ | |||
|
|
|||
| 在深度学习模型的训练过程中，通过梯度下降算法更新网络参数。一般地，梯度下降算法分为前向传播和反向更新两个阶段。 | |||
|
|
|||
| - 在前向传播阶段，输入向量使用下列公式，从前往后，计算下一层每个神经元的值。其中，O为神经元的输入和输出，f为激活函数，W为权重，b为偏置。 | |||
| 在 **前向传播阶段** ，输入向量使用下列公式，从前往后，计算下一层每个神经元的值。其中，O为神经元的输入和输出，f为激活函数，W为权重，b为偏置。 | |||
There was a problem hiding this comment.
请注意：O,f,W,b用数学公式，保持上下文一致。
| @@ -142,7 +145,7 @@ | |||
| \\global\_norm=\sqrt{\sum_{i=0}^{n-1}(norm(X[i]))^2}\\ | |||
|
|
|||
|
|
|||
| 其中 :math:`X_i` 为梯度向量，clip_norm 为设置的L2范数阈值， :math:`norm（X）` 代表 :math:`X` 的L2范数，global_norm 为所有梯度向量的L2范数的均方根值。 | |||
| :math:`X_i` 为梯度向量，clip_norm 为设置的L2范数阈值， :math:`norm（X）` 代表 :math:`X` 的L2范数，global_norm 为所有梯度向量的L2范数的均方根值。 | |||
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
优化梯度裁剪文档。