* [Fleet] Improve the quick start fo fleet colletive

* [Fleet] Some minor changes

* [Fleet] Modify some parts

* Update cluster_quick_start_collective_cn.rst

* [Fleet] Modify the quick start based on the reviews

* [Fleet] Use the new launch command

* [Fleet] Chage the version requirement

* [Fleet] Change the title

Co-authored-by: liuTINA0907 <[email protected]>

the doc was editted based on the discussion in the issue: 
INT32 Failed on paddle.metric.accuracy #42845
PaddlePaddle/Paddle#42845

* add distributed deployment

* update index

* review and update

* add faq

* translate launch api doc and fix doc by comments

* add ps quick start

* dataset and contents

* update title

* reader

* update dataset

* Update index_cn.rst

* Update index_cn.rst

Co-authored-by: Chen Long <[email protected]>

* Update release_note_cn.md

* Update release_note_en.md

* Update release_note_en.md

* Update release_note_cn.md

* Update release_note_cn.md

* Update release_note_en.md

* fix api docs bug

* update AlexNet docs

* update

* update

* update

* Update auto_cast_cn.rst

* Update save_cn.rst

* Update auto_cast_cn.rst

* Update auto_cast_cn.rst

* Update save_cn.rst

* Update auto_cast_cn.rst

Co-authored-by: Ligoml <[email protected]>

* add nanmedian doc

* Update nanmedian_cn.rst

Co-authored-by: Ligoml <[email protected]>

* align the code example with en docs using copy from function

* update RandomCrop api

* update multiplex,reshape,put(take)_alone_axis api to resolve the conflict

* update format issue for randomcrop api

* fix format issue

* fix format issue

* fix format issue

* use copy-from function for the code example

* update uniform_cn.rst

* Update uniform_cn.rst

* Update RandomCrop_cn.rst

Co-authored-by: Ligoml <[email protected]>

* add cn doc for rrelu

* fix doc format

* fix doc format

* fix paper link

* 参数加粗

* fix typo prelu

* Update bernoulli_cn.rst

* Update bernoulli_cn.rst

* Update bernoulli_cn.rst

* Update bernoulli_cn.rst

* Update ones_cn.rst

* Update ones_cn.rst

* Add prefetch_factor

* Update DataLoader_cn.rst

* Update DataLoader_cn.rst

* update custom cpp op doc

* revert shape change

* fix typo

* fix  number error

* replace with paddle api

* fix conv3d

* update doc

* Update assign_cn.rst

* Update AdaptiveAvgPool1D_cn.rst

* Update adaptive_avg_pool1d_cn.rst

* Update XavierNormal_cn.rst

* Update XavierUniform_cn.rst

* Update where_cn.rst

* Update AdaptiveAvgPool1D_cn.rst

* Update AdaptiveAvgPool1D_cn.rst

* Update XavierNormal_cn.rst

* Update XavierUniform_cn.rst

* Update where_cn.rst

* Update AdaptiveAvgPool1D_cn.rst

* Update adaptive_avg_pool1d_cn.rst

* 汇聚->池化

* Update AdaptiveAvgPool1D_cn.rst

Co-authored-by: Ligoml <[email protected]>

* add docs of CyclicLR

* fix typo

* update

* update

* test

* test

* test

* add default value

* change style

* change more

* fix punctuation error

…4737)

* add docs for API CosineEmbeddingLoss

* aligning to English document

* add name parameter

* align the code example with en docs using copy from function

* update full and sort api

* resolve the conflict for multiplex,put_alone_axis, and reshape api

* resolve the conflict for take_alone_axis api cn docs

* Update multiplex_cn.rst

* Update multiplex_cn.rst

* Update put_along_axis_cn.rst

* Update put_along_axis_cn.rst

* Update sort_cn.rst

* Update take_along_axis_cn.rst

Co-authored-by: Ligoml <[email protected]>

* Create TripletMarginLoss_cn.rst

* Update Overview_cn.rst

* Update TripletMarginLoss_cn.rst

* Create triplet_margin_loss_cn.rst

* Update TripletMarginLoss_cn.rst

* Update Overview_cn.rst

* Update triplet_margin_loss_cn.rst

* Update triplet_margin_loss_cn.rst

* Update TripletMarginLoss_cn.rst

* Update TripletMarginLoss_cn.rst

* Update triplet_margin_loss_cn.rst

* Update TripletMarginLoss_cn.rst

* Update triplet_margin_loss_cn.rst

* Update triplet_margin_loss_cn.rst

* Update triplet_margin_loss_cn.rst

* Update Overview_cn.rst

* Update TripletMarginLoss_cn.rst

* Update triplet_margin_loss_cn.rst

* Update TripletMarginLoss_cn.rst

* Update TripletMarginLoss_cn.rst

* Update triplet_margin_loss_cn.rst

* Update TripletMarginLoss_cn.rst

* Update triplet_margin_loss_cn.rst

* Update TripletMarginLoss_cn.rst

* Update triplet_margin_loss_cn.rst

Co-authored-by: Ligoml <[email protected]>

* Create TripletMarginWithDistanceLoss_cn.rst

* Update Overview_cn.rst

* Update Overview_cn.rst

* Update TripletMarginWithDistanceLoss_cn.rst

* Create triplet_margin_with_distance_loss_cn.rst

* Update TripletMarginWithDistanceLoss_cn.rst

* Update triplet_margin_with_distance_loss_cn.rst

* Update TripletMarginWithDistanceLoss_cn.rst

* Update triplet_margin_with_distance_loss_cn.rst

* Update TripletMarginWithDistanceLoss_cn.rst

* Update triplet_margin_with_distance_loss_cn.rst

* Update triplet_margin_with_distance_loss_cn.rst

* Update TripletMarginWithDistanceLoss_cn.rst

* Update TripletMarginWithDistanceLoss_cn.rst

* Update triplet_margin_with_distance_loss_cn.rst

* Update TripletMarginWithDistanceLoss_cn.rst

* Update triplet_margin_with_distance_loss_cn.rst

* Update triplet_margin_with_distance_loss_cn.rst

* Update triplet_margin_with_distance_loss_cn.rst

* Update TripletMarginWithDistanceLoss_cn.rst

* Update Overview_cn.rst

* Update TripletMarginWithDistanceLoss_cn.rst

* Update triplet_margin_with_distance_loss_cn.rst

* Update triplet_margin_with_distance_loss_cn.rst

Co-authored-by: Ligoml <[email protected]>

* add op optimization file

* add op optimization file

* add new images

* add some changes according to review

* add some changes according to review

* Update op_optimization_accpetance_criteria_cn.md

* Update op_optimization_contributing_guides_cn.rst

* revise accroding to the dirname changes in community repo

* Update op_optimization_contributing_guides_cn.rst

Co-authored-by: Chen Long <[email protected]>

* add moe docs

* update

* update

* update

* update_name_typos

* Update fft2_cn.rst

* Update fft_cn.rst

* Update fftfreq_cn.rst

* Update fftn_cn.rst

* Update kldiv_loss_cn.rst

* Update spectral_norm_cn.rst

* Update fftshift_cn.rst

* Update rfftn_cn.rst

* Update rfftfreq_cn.rst

* Update rfft_cn.rst

* Update rfft2_cn.rst

* Update irfftn_cn.rst

* Update irfft2_cn.rst

* Update ihfftn_cn.rst

* Update irfft_cn.rst

* Update hfft2_cn.rst

* Update hfft_cn.rst

* Update ifft2_cn.rst

* Update ifftn_cn.rst

* Update ihfft2_cn.rst

* Update hfftn_cn.rst

* Update ifft_cn.rst

* Update ifftshift_cn.rst

* Update ihfft_cn.rst

* Update remove_weight_norm_cn.rst

* Update weight_norm_cn.rst

* Update RecordEvent_cn.rst

* Update reshape_cn.rst

* Update load_cn.rst

* Update istft_cn.rst

* Update stft_cn.rst

* Update Layer_cn.rst

* Update Auc_cn.rst

* Update Recall_cn.rst

* Update Precision_cn.rst

Co-authored-by: Chen Long <[email protected]>

* refine paddle.amp.decorate input parameter of optimizer

* Revert "refine paddle.amp.decorate input parameter of optimizer"

This reverts commit 2899b2a.

* refine cn doc for state_dict

* refine doc

* refine doc

* add paddle.static.set_ipu_shard

* up 0

* rename file

* use full-width comma

* add final newline

* fix typos: CRS -> CSR

* remove ResNeXt class API from 2.3.0 release note

* refine cn docs

* width_per_group -> width

* remove ResNeXt from Overview

* trigger

* copy from en docs

* add a blank line

* trigger ci (remove PADDLEPADDLE_PR)

* refine `Returns`

* add space between cn and en char

* refine desc of pretrained

* fix wrong ref

* modify comma

* modify stop

* modify stop after None

* modify stop

* stop after serial num

* stop in the end

* 。 between nums

* colon 1

* :xxx:`xxx`

* url fix

* semicolon

* Update GradScaler_cn.rst

* Update EarlyStopping_cn.rst

* Update ParallelEnv_cn.rst

* Update Transform_cn.rst

* Update fft2_cn.rst

* Update fftfreq_cn.rst

* Delete ResNeXt_cn.rst

* Update ResNet_cn.rst

Co-authored-by: Ligoml <[email protected]>

* use copy-from instead of copy, test=document_fix

* fix log_freq default value (PaddlePaddle/Paddle#43343), test=document_fix

* add space between cn and en char, test=document_fix

* add `optional`, test=document_fix

* add missing parameters, test=document_fix

* remove extra parameters, test=document_fix

* add missing links for `ProgBarLogger` and `ModelCheckpoint`, test=document_fix

* update stack_outputs desc, test=document_fix

* None -> 无, test=document_fix

* fix some parameters, test=document_fix

* resolve conflicts

…ls.alexnet, paddle.vision.models.mobilenet, ops apis (#4704)

* Update Normalize_cn.rst

* Update Normalize_cn.rst

* Update alexnet_cn.rst

* Update VGG_cn.rst

* Update MobileNetV2_cn.rst

* Update Normalize_cn.rst

* Update MobileNetV1_cn.rst

* Update MobileNetV1_cn.rst

* Update MobileNetV2_cn.rst

* Update VGG_cn.rst

* Update PSRoIPool_cn.rst

* Update psroi_pool_cn.rst

* Update roi_align_cn.rst

* Update RoIAlign_cn.rst

* Update alexnet_cn.rst

* Update MobileNetV1_cn.rst

* Update MobileNetV2_cn.rst

* Update VGG_cn.rst

* Update VGG_cn.rst

* Update PSRoIPool_cn.rst

* Update psroi_pool_cn.rst

* Update psroi_pool_cn.rst

* Update roi_align_cn.rst

* Update psroi_pool_cn.rst

* Update roi_align_cn.rst

* Update Normalize_cn.rst

* Update VGG_cn.rst

* Update alexnet_cn.rst

* Update PSRoIPool_cn.rst

* Update docs/api/paddle/vision/models/alexnet_cn.rst

Co-authored-by: Nyakku Shigure <[email protected]>

Co-authored-by: Ligoml <[email protected]>
Co-authored-by: Nyakku Shigure <[email protected]>

* Update calculate_gain_cn.rst

* Update Constant_cn.rst

* Update KaimingNormal_cn.rst

* Update KaimingUniform_cn.rst

* Update KaimingNormal_cn.rst

* Update randint_cn.rst

* Update randint_cn.rst

* Update squeeze_cn.rst

* Update argmin_cn.rst

* Update add_n_cn.rst

* Update add_n_cn.rst

* Update unique_cn.rst

* Update topk_cn.rst

* Update argmin_cn.rst

* Update argmin_cn.rst

* Update Constant_cn.rst

* Update calculate_gain_cn.rst

* Update squeeze_cn.rst

* Update topk_cn.rst

* Update unique_cn.rst

* Update unique_cn.rst

* Update topk_cn.rst

* Update add_n_cn.rst

* Update argmin_cn.rst

* Update Constant_cn.rst

* Update KaimingNormal_cn.rst

* Update KaimingUniform_cn.rst

* Update KaimingNormal_cn.rst

* Update KaimingUniform_cn.rst

* Update randint_cn.rst

* Update squeeze_cn.rst

* Update topk_cn.rst

* Update add_n_cn.rst

* Update Constant_cn.rst

Co-authored-by: Ligoml <[email protected]>

* ps overview

* update

* ps overview background

* ps overview api

* save load

* update codestyle

* Update cluster_overview_ps_cn.rst

* Update cluster_overview_ps_cn.rst

* add picture

* update online learning

* online learning

* update

* update quick start

* update ps quick start

* update loss

* update introduction

Co-authored-by: wuzhihua <[email protected]>

…ource code (#4703)

* Fix the problem that the parameter content is inconsistent with the source code

Fix the problem that the parameter content is inconsistent with the source code

* Modify the document parameters and add a period to the document.

Modify the document parameters and add a period to the document.

* fix

* fix

* fix

* fix again

* rollback

* Update load_cn.rst

* Update list_cn.rst

* Update help_cn.rst

Co-authored-by: Ligoml <[email protected]>

* add method introduction for op-perf

* add some changes

* add some changes

* change according to the advices

* Update release_note_cn.md

* Update release_note_en.md

* Update release_note_en.md

* Update release_note_cn.md

* Update release_note_cn.md

* Update release_note_en.md

* mv kp index

* fix image url

* update paddle.distributed.launch with IPU

* update doc

* fix conv3d_transpose doc

* modify docs of conv3d_transpose

* Create MultiLabelSoftMarginLoss_cn.rst

* Update Overview_cn.rst

* Update MultiLabelSoftMarginLoss_cn.rst

* Create multi_label_soft_margin_loss_cn.rst

* Update MultiLabelSoftMarginLoss_cn.rst

* Update MultiLabelSoftMarginLoss_cn.rst

* Update multi_label_soft_margin_loss_cn.rst

* Update MultiLabelSoftMarginLoss_cn.rst

* Update MultiLabelSoftMarginLoss_cn.rst

* Update MultiLabelSoftMarginLoss_cn.rst

* Update multi_label_soft_margin_loss_cn.rst

* Update multi_label_soft_margin_loss_cn.rst

* Update multi_label_soft_margin_loss_cn.rst

* modify new_cpp_ op doc

* fix format

* add invoke

* fix picture

* update

* fix format

* fix code format

* Update new_cpp_op_cn.md

* Update new_cpp_op_cn.md

* Update new_cpp_op_cn.md

Co-authored-by: Chen Long <[email protected]>

* begin a pr (just a placeholder)

* add a blank line

* add space between cn and en char

* "sing `COPY-FROM`

* add a blank line between h2 and p

* refine returns

* fix the case of proper

* refine parameter `pretrained`

* add parameter `kwargs`

* refine parameter `num_classes`

* missing parameter `with_pool`

* CRLF -> LF

@Ainavo 出来挨打，为什么你写的全是 CRLF 的……

* add some missing api labels

* `默认值：` -> `默认值为 `

* refine returns

* :math:\`0.0\`

* begin a pr (just a placeholder), test=document_fix

* refine arguments

* refine api desc

* fix wrong COPY-FROM label in Model docs

* add some returns

* `默认值：` -> `默认值为 `

* refine returns

* COPY-FROM

* add a blank line in code block

* Update MSRAInitializer_cn.rst

* Update MSRAInitializer_cn.rst

* Update MSRAInitializer_cn.rst

* Update MSRAInitializer_cn.rst

* Update KaimingNormal_cn.rst

* Update KaimingUniform_cn.rst

* Update KaimingNormal_cn.rst

* Update MSRAInitializer_cn.rst

* Update KaimingNormal_cn.rst

* Update KaimingUniform_cn.rst

* Update KaimingNormal_cn.rst

* Update KaimingUniform_cn.rst

Co-authored-by: Ligoml <[email protected]>

* update doc for AD

* format modify

* add introduction for new AD

* fix small bug

* modify for ref link

* Update autograd_cn.rst

Co-authored-by: Chen Long <[email protected]>

* refine paddle.amp.decorate input parameter of optimizer

* Revert "refine paddle.amp.decorate input parameter of optimizer"

This reverts commit 2899b2a.

* refine amp doc

* refine doc

* refine english doc

* refine amp doc

* refine cn

* refine cn

* refine en

* refine code

* refine doc

* Update amp_cn.md

1、## 一、概述：
->去掉冒号
2、通过`nvidia-smi`指令可帮助查看NVIDIA显卡架构信息
->改为：通过`nvidia-smi`指令可帮助查看NVIDIA显卡架构信息，混合精度训练适用的 NVIDIA GPU 计算能力至少为 7.0 的版本。
3、分布式训练和动转静因为用法和动态图相似，不单独列子标题了，改为一段话即可：
-> 前文介绍了动态图模式下单卡（GPU）训练的方法，与之类似，[分布式训练](https://fleet-x.readthedocs.io/en/latest/paddle_fleet_rst/collective/collective_performance/amp.html) 和 [动转静训练](../jit/index_cn.html) 时可以采用同样的方法开启 AMP。接下来主要介绍不同的静态图模式下开启 AMP 训练的方法，以及 AMP 训练的进阶用法，如梯度累加。
4、静态图 AMP 的功能逻辑与动态图类似，只是调用的接口有区别
->加上这句话

* refine cn dic

* refine en doc

* refine doc

Co-authored-by: moguguo <[email protected]>

* add pre-commit guide

* add pre_commit doc in index

* temp

* modify according to chenlong's comment

* modify index,dev_guide,ci_manual

* update new_cpp_op

* fix format

* improve gradient_clip doc

* modify according to zhouwei's comment

* modify according to zhangkeming's comment

* modify according to chenlong's comment

* refine paddle.amp.decorate input parameter of optimizer

* Revert "refine paddle.amp.decorate input parameter of optimizer"

This reverts commit 2899b2a.

* fix picture path

* refine code

* refine code

* refine code

* refine code

* doc(): add IPU dy2static related docs.

* fix(): fix comments.

* fix(): add int type doc for reduction.

* fix(): fix typo.

* fix(): fix order.

* fix(): fix comments.

* fix(): adjust mean and sum in identity_loss.

* move move API to paddle.incubate.identity_loss

Co-authored-by: Allen Guo <[email protected]>

* Update release_note_cn.md

* Update release_note_en.md

* Update release_note_en.md

* Update release_note_cn.md

* Update release_note_cn.md

* Update release_note_en.md

* add 2.3.1 release note

* Update release_note_cn.md

Co-authored-by: jzhang533 <[email protected]>

* [Dy2St] Polish basic usage document of @to_static

* add principle doc

* Update release_note_cn.md

* Update release_note_en.md

* Update release_note_en.md

* Update release_note_cn.md

* Update release_note_cn.md

* Update release_note_en.md

* rm TCChenlong from ci approval list

* modify clip

* modify variable presentation

* op dtype extension

* rename op_dtype_extension_contributing_guides

* doc preview test

* Update op_dtype_extension_contributing_guides_cn.rst

* Create soft_margin_loss_cn.rst

* Create SoftMarginLoss_cn.rst

* Update SoftMarginLoss_cn.rst

* Update soft_margin_loss_cn.rst

* Update SoftMarginLoss_cn.rst

* Update Overview_cn.rst

* Update soft_margin_loss_cn.rst

* Update soft_margin_loss_cn.rst

* Update SoftMarginLoss_cn.rst

* Update soft_margin_loss_cn.rst

Co-authored-by: Ligoml <[email protected]>

* make the word case correct

* `brew install unrar` -> `brew install rar`

* `homebrew` -> `Homebrew`

* fix typo (PyTroch -> PyTorch) (PaddlePaddle/Paddle#44042)

* try manually run `pre-commit` to avoid CI error

* typo (`P ython`)

* remove docs/tutorial

#5087)

* add cn docs for all_gather_object, modified docs for all_gather and split

* add warning

* fix all gather doc

* add overview

* add doc for count_nonzero

* fix format error

* Update count_nonzero_cn.rst

Co-authored-by: Ligoml <[email protected]>

* docs

* fix some style issue by docufix

* refine docs

* translate `pairwise` to `两两之间`

Co-authored-by: SigureMo <[email protected]>

* Create instalL_NGC_PaddlePaddle_ch.rst

one page on installing instruction on NGC PaddlePaddle was created.

* Update index_cn.rst

add a file link to NGC PaddlePaddle installation page

* Update instalL_NGC_PaddlePaddle_ch.rst

fix 2 errors：remove one link that is not needed, and correct the link for NVIDIA 容器工具包

* Update instalL_NGC_PaddlePaddle_ch.rst

* Update instalL_NGC_PaddlePaddle_ch.rst

update the head footnote

* Update instalL_NGC_PaddlePaddle_ch.rst

remove the numbers in headline, in order to make the doc align with others

* Update instalL_NGC_PaddlePaddle_ch.rst

try to fix a link issue

* docs bugs fix

* Update numel_cn.rst

Co-authored-by: Ligoml <[email protected]>

* modify_save_load

* fix image path

* update doc of new python api

* fix some doc

* update

* update dygraph mode

* modify with review