merge newest Paddle code #3

thisjiang · 2021-01-07T09:32:28Z

PR types

PR changes

Describe

* enable bilateral_slice unittest on windows platform * reduce max threads

* update, test=develop

* split callback unittest * rm test_callback from timeout list

* fix conv_transpose bug when padding=same

…29574)

…or (#29842) * Revert "[inplace] Add ShareHolderWith for class Variable and SharePlaceholderWith in VarBase.detach() to share the same Tensor/SelectedRows (#29267)" This reverts commit b10ecd9. * Support ShareInplaceVersionCounterWith to share the same inplace version counter for VarBase

* register op version for grid_sampler

* Support MIPS arch.

… executor (#29926)

* fix a bug in reset_tensor_array, test=develop * ci coverage, test=develop

…29923) * add op version for fake_quant and fake_dequant ops, test=op_version, test=develop

…remove unnecessay code (#29851)

* flush denormals to zero, test=develop * add comments, test=develop

* rm check_import_scipy, rm chunk_eval and mean_iou in paddle.metric.__init__.py * Revert "rm check_import_scipy, rm chunk_eval and mean_iou in paddle.metric.__init__.py" This reverts commit 179ba8c. * delete paddle.metric.chunk_eval and paddle.metric.mean_iou * delete paddle.nn.clip and paddle.nn.clip_by_norm * delete paddle.nn.functional.activation.hard_sigmoid and paddle.nn.functional.activation.hard_swish * delete paddle.nn.Pool2D, paddle.nn.BilinearTensorProduct, paddle.nn.RowConv, paddle.nn.functional.row_conv * fix extension import error * fix unittest for row_conv and Pool2D

* add gru op_register_version; test=op_version; * Update fc,mul version;test=op_version;

* add xpu_coverage function * xpu coverage ipipe only deal with xpu files * fix import error * fix format error * 'fix format error' * fix format error * fix error * fix format error * fix format error

fix the state_dict bug for the xpu

PADDLE_RETRY_CUDA_SUCCESS used wrong sleep time so it can cause timeout in unittest. This PR fixed it. After we searched the doc in https://pubs.opengroup.org/onlinepubs/7908799/xsh/unistd.h.html, the time unit of sleep in unistd.h takes "seconds", usleep takes "microseconds", Sleep in windows.h takes "milliseconds".

* map matmul/squeeze2+matmul/reshape2+matmul to mul

… executor (#29961)

test=develop test=op_version

* fix cv2 rotation

…ise_conv2d_transpose, test=op_version (#29937)

* snap * add inference api: DisableTensorRtOPs * fix code style * update api to experimental * update variable name

* add aarch64 and sunway kunlun lib * minor * optimize elementwise_add for kunlun * update kunlun dependence * minor * minor

* Polish and Optimize the print/repr message of all layer * fix some code format

* add dispenable input 'shape' for core.ops.reshape2 * add dispenable inputs for core.ops.reshape2/expand/slice * add ut

* fix syncbn convet * add unittest

* test=develop, add model_average and lookahead

Change-Id: Ia5279b0cbb6a5b3970aff66e9510e0d85efa70ce

* Simplify the options of spawn based on fleetrun * polish details * polish doc details

… introduced precision diff. test=develop (#30122)

* enhance error info for py_func * update

…erStatus_t (#30161)

* test=develop, add index_select_cuda kernel

* initial commit: simple demo * polish copyright format * add grap op simple demo * adapt uncertain number of argument * change trait marco name * add place & dtype support for add kernel * add dispath and infershape func * poish code & add notes * add dynamic_loader dep for paddle_framework * add new custom op test dir * polish impl details * add unittest for new custom op * fix failed unittest * Costum op (#1) * fix compile error * wrap framework tensor with LoDTensor * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * add CustomTensor default constructor * add size() for CustomTensor * make size const for CustomTensor * refactor place related api to circle the concept * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * make place const * make Tensor copy * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * remove additional head of framework * use back to shared ptr for custom tensor * use back to shared ptr for custom tensor * use back to shared ptr for custom tensor * use back to shared ptr for custom tensor * use back to shared ptr for custom tensor * use back to shared ptr for custom tensor * add gpu test * merge latest cwh code in * adjust ut code of custom op * adjust ut code of custom op * adjust ut code of custom op * Remove ShareData from user && Change CustomTensor to Tensor && Support more data type (#2) * fix compile error * wrap framework tensor with LoDTensor * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * add CustomTensor default constructor * add size() for CustomTensor * make size const for CustomTensor * refactor place related api to circle the concept * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * make place const * make Tensor copy * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * remove additional head of framework * use back to shared ptr for custom tensor * use back to shared ptr for custom tensor * use back to shared ptr for custom tensor * use back to shared ptr for custom tensor * use back to shared ptr for custom tensor * use back to shared ptr for custom tensor * add gpu test * merge latest cwh code in * adjust ut code of custom op * adjust ut code of custom op * adjust ut code of custom op * adjust ut code of custom op * adjust ut code of custom op * hid share data from and to * rename CustomTensor to Tensor * refactor register design & add test * change op_funtion to op_meta_info * split op meta info into .h and .cc * move get methods into friend class * move OpMetaInfoHelper into framework space * move CustomTensorUtils into framework space * change pybind api name * move PD C API into op meta info * add register custom op api * remove inference cmake change * refactor copy to api && change Reshape to lowercase && support more dtype && add more test (#3) * fix compile error * wrap framework tensor with LoDTensor * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * add CustomTensor default constructor * add size() for CustomTensor * make size const for CustomTensor * refactor place related api to circle the concept * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * make place const * make Tensor copy * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * remove additional head of framework * use back to shared ptr for custom tensor * use back to shared ptr for custom tensor * use back to shared ptr for custom tensor * use back to shared ptr for custom tensor * use back to shared ptr for custom tensor * use back to shared ptr for custom tensor * add gpu test * merge latest cwh code in * adjust ut code of custom op * adjust ut code of custom op * adjust ut code of custom op * adjust ut code of custom op * adjust ut code of custom op * hid share data from and to * rename CustomTensor to Tensor * support multi dtype * remove lod, make reshape lowercase, add copy test and refactor copy api * remove lod, make reshape lowercase, add copy test and refactor copy api * remove lod, make reshape lowercase, add copy test and refactor copy api * remove lod, make reshape lowercase, add copy test and refactor copy api * fix copy to error * add more test * add more test * add more test * add more test * add more test * add more test * add more test * add more test * add more test * add more test * add more test * add more test * add more test * add more test * add more test * add more test * polish detail & error message * polish test details * Add cast api && Change copy related api to copy_to && add more test (PaddlePaddle#4) * fix compile error * wrap framework tensor with LoDTensor * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * add CustomTensor default constructor * add size() for CustomTensor * make size const for CustomTensor * refactor place related api to circle the concept * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * fix compile error * make place const * make Tensor copy * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * debug CustomTensor core * remove additional head of framework * use back to shared ptr for custom tensor * use back to shared ptr for custom tensor * use back to shared ptr for custom tensor * use back to shared ptr for custom tensor * use back to shared ptr for custom tensor * use back to shared ptr for custom tensor * add gpu test * merge latest cwh code in * adjust ut code of custom op * adjust ut code of custom op * adjust ut code of custom op * adjust ut code of custom op * adjust ut code of custom op * hid share data from and to * rename CustomTensor to Tensor * support multi dtype * remove lod, make reshape lowercase, add copy test and refactor copy api * remove lod, make reshape lowercase, add copy test and refactor copy api * remove lod, make reshape lowercase, add copy test and refactor copy api * remove lod, make reshape lowercase, add copy test and refactor copy api * fix copy to error * add more test * add more test * add more test * add more test * add more test * add more test * add more test * add more test * add more test * add more test * add more test * add more test * add more test * add more test * add more test * add more test * add type cast * add cast and make copy to api * add cast and make copy to api * add cast and make copy to api * add cast and make copy to api * merge cwh code * merge cwh code * merge cwh code * merge cwh code * merge cwh code * add more error log * add more error log * polish code * used for test * remove test comment * remove test comment * fix uint8 type error * fix lost uint8 type error * add test for coverage * polish details by reviewer comments * add prefix for DISABLE_COPY_AND_ASSIGN Co-authored-by: Jiabin Yang <[email protected]>

Init ir printer

…ten::DenseTensor, test=allcases (PaddlePaddle#38473) * Added shared_ptr<Allocation> member & corresponding interfaces to Storage * Removed original pten::Allocation from Storage and adjusted the interfaces accordingly * Fixed issues with storage offset * Used place to malloc allocation for TensorStorage * [Unify Tensors PR #3]Ported framework::Tensor interfaces to pten::DenseTensor * Fixed issues with place * Added comments * Moved mutable_data with stream argument to DenseTensor * Added set_offset interface * Fixed CI issues,test=allcases * [Unify Tensors PR PaddlePaddle#4] Port LoDTensor interfaces to DenseTensor * Reverted changes too pten_layout() interface * Removed friend classes

…seTensor,test=allcases (PaddlePaddle#38632) * Added shared_ptr<Allocation> member & corresponding interfaces to Storage * Removed original pten::Allocation from Storage and adjusted the interfaces accordingly * Fixed issues with storage offset * Used place to malloc allocation for TensorStorage * [Unify Tensors PR #3]Ported framework::Tensor interfaces to pten::DenseTensor * Fixed issues with place * Added comments * Moved mutable_data with stream argument to DenseTensor * Added set_offset interface * Fixed CI issues,test=allcases * [Unify Tensors PR PaddlePaddle#4] Port LoDTensor interfaces to DenseTensor * Removed friend class EigenTensor/EigenMatrix/EigenVector from Tensor * Modified framework::Tensor to inherit from DenseTensor * Reverted changes too pten_layout() interface * Removed friend classes * Rearranged cfunction calls from tensor.data<void>() to tensor.data() * Fixed CI issues * Fixed lite issues * Fixed data() interface issues,test=allcases * Resolved IsInitialized() issues * Fixed ResetHolder() issues * Fixed MKLDNN & Storage issues * Resolved ShareBufferWith() issues * Fixed LoD issues

…od_tensor,test=allcases (PaddlePaddle#38811) * Added shared_ptr<Allocation> member & corresponding interfaces to Storage * Removed original pten::Allocation from Storage and adjusted the interfaces accordingly * Fixed issues with storage offset * Used place to malloc allocation for TensorStorage * [Unify Tensors PR #3]Ported framework::Tensor interfaces to pten::DenseTensor * Fixed issues with place * Added comments * Moved mutable_data with stream argument to DenseTensor * Added set_offset interface * Fixed CI issues,test=allcases * [Unify Tensors PR PaddlePaddle#4] Port LoDTensor interfaces to DenseTensor * Removed friend class EigenTensor/EigenMatrix/EigenVector from Tensor * Modified framework::Tensor to inherit from DenseTensor * Reverted changes too pten_layout() interface * Removed friend classes * Rearranged cfunction calls from tensor.data<void>() to tensor.data() * Fixed CI issues * Fixed lite issues * Fixed data() interface issues,test=allcases * Resolved IsInitialized() issues * Fixed ResetHolder() issues * Fixed MKLDNN & Storage issues * Resolved ShareBufferWith() issues * Fixed LoD issues * Removed interfaces & members from lod_tensor,test=allcases

PaddlePaddle#39128) * Added selected_rows and rw_lock to pten * Renamed the unit test target to fix CI * Removed Class SelectedRows in Fluid, changed include/cmake relationship, use pten::SelectedRows in Fluid * Remove rw_lock.h,rw_lock_test.cc in fluid * Use pten::RWLock and pten::AutoRDLock, fix CI * Use pten::SelectedRows * Use pten::SelectedRows * Fix to pass NPU CI * Use pten::SelectedRows, to pass NPU CI * To fix NPU CI * To fix NPU CI again

LielinJiang and others added 30 commits December 25, 2020 19:19

Enable bilateral_slice unittest on windows platform (#29896)

0f4b218

* enable bilateral_slice unittest on windows platform * reduce max threads

fix the bug in pipeline data parallelism (#29731)

01950ce

* update, test=develop

Split callbacks unittest (#29914)

11de384

* split callback unittest * rm test_callback from timeout list

Fix Conv2DTanspose bug when padding='same' (#29915)

0b74428

* fix conv_transpose bug when padding=same

[Kunlun] PR2: Support MultiDevicePass and BKCL in parallel executor (#…

4427df3

…29574)

fix optimizer dtype (#29917)

a4b9daf

fix reg api ut fail (#29921)

5c162fe

Register op version for grid_sampler, test=op_version (#29916)

eab0b60

* register op version for grid_sampler

Support mips arch (#29903)

332da13

* Support MIPS arch.

[Kunlun] bug fix of PR2: Support MultiDevicePass and BKCL in parallel…

3d1741b

… executor (#29926)

fix a bug in reset_tensor_array, test=develop (#29620)

acb5e86

* fix a bug in reset_tensor_array, test=develop * ci coverage, test=develop

add op version for fake_quant and fake_dequant ops, test=op_version (#…

7667e59

…29923) * add op version for fake_quant and fake_dequant ops, test=op_version, test=develop

[Windows CI test] Enable unittest test_optimizer_in_control_flow and …

14bd77f

…remove unnecessay code (#29851)

flush denormals to zero, test=develop (#29924)

181ea18

* flush denormals to zero, test=develop * add comments, test=develop

add bkcl.so in whl for kunlun (#29947)

f4be9d6

Register op version for print, test=op_version (#29945)

913f77a

[Inference] Solve 2.0 trt performance reduce compare 1.8. (#29925)

2b1d796

add gru op_register_version; test=op_version; (#29931)

5a4e42c

* add gru op_register_version; test=op_version; * Update fc,mul version;test=op_version;

Support xpu ut coverage (#29892)

121658d

* add xpu_coverage function * xpu coverage ipipe only deal with xpu files * fix import error * fix format error * 'fix format error' * fix format error * fix error * fix format error * fix format error

fix the state_dict bug for the xpu (#29888)

587b67e

fix the state_dict bug for the xpu

map matmul/squeeze2+matmul/reshape2+matmul to mul (#29911)

6a0102b

* map matmul/squeeze2+matmul/reshape2+matmul to mul

[Kunlun] bug fix of PR2: Support MultiDevicePass and BKCL in parallel…

bb20dcf

… executor (#29961)

support grad accumulated across batch (#29942)

a1d9a14

Register op version for coalesce_tensor. (#29940)

6ac4f0a

test=develop test=op_version

compile the denormal.cc on aarch64, test=develop (#29956)

9586122

Fix rotation bug when use cv2 backend (#29933)

ec2fad4

* fix cv2 rotation

register op version for conv2d_transpose, conv3d_transpose and depthw…

be8b5fd

…ise_conv2d_transpose, test=op_version (#29937)

shangzhizhou and others added 23 commits January 6, 2021 15:54

add inference api： DisableTensorRtOps (#30109)

05b2769

* snap * add inference api: DisableTensorRtOPs * fix code style * update api to experimental * update variable name

add aarch64 and sunway kunlun lib (#30027)

8e1c3dd

* add aarch64 and sunway kunlun lib * minor * optimize elementwise_add for kunlun * update kunlun dependence * minor * minor

fix error message for distribute_fpn_proposals_op (#30116)

69839f8

fix unittest failed on windows (#29837)

9c99d37

Extend the timeout for the (#30151)

f3a2392

Polish and Optimize the print/repr information of Layer (#29998)

30888ca

* Polish and Optimize the print/repr message of all layer * fix some code format

fix error message (#30135)

becf99d

fix ubuntu18 openssl error (#30077)

35fbc48

update readme test=document_fix (#30154)

3be6593

add dispenable input for core.ops.reshape2/expand/slice (#30072)

adac38c

* add dispenable input 'shape' for core.ops.reshape2 * add dispenable inputs for core.ops.reshape2/expand/slice * add ut

fix syncbn convert (#30158)

6a19e41

* fix syncbn convet * add unittest

Add Lookahead and ModelAverage Optimizer (#30004)

198fbdf

* test=develop, add model_average and lookahead

pre padding in dygraph (#30163)

4763e6b

Change-Id: Ia5279b0cbb6a5b3970aff66e9510e0d85efa70ce

Simplify the options of spawn based on fleetrun (#30144)

8020e34

* Simplify the options of spawn based on fleetrun * polish details * polish doc details

fix enforce msg of sum xpu op (#30113)

f5428ec

fix assign_op_xpu concat_op_xpu warining (#30120)

15fac5e

[XPU] Remove lite_xpu ut lite_resnet50_test since fusion pass changes…

b8207af

… introduced precision diff. test=develop (#30122)

open normal unittest on windows (#30167)

6aa82e0

enhance error info for py_func (#30138)

91a8a25

* enhance error info for py_func * update

Add detailed error message for curandStatus_t, cublasStatus_t, cusolv…

404c167

…erStatus_t (#30161)

refine the paddle place support using str (#28769)

7dd551e

Improve Index select cuda kernel (#30139)

c5b415b

* test=develop, add index_select_cuda kernel

fix paddle.pow doc, test=document_fix (#30159)

1ff69f5

thisjiang merged commit 84eb899 into thisjiang:develop Jan 7, 2021

thisjiang pushed a commit that referenced this pull request Oct 28, 2021

Merge pull request #3 from Superjomn/init-ir-printer

56bba20

Init ir printer

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

merge newest Paddle code #3

merge newest Paddle code #3

thisjiang commented Jan 7, 2021

merge newest Paddle code #3

merge newest Paddle code #3

Conversation

thisjiang commented Jan 7, 2021

PR types

PR changes

Describe