Custom Operator Random Number Generator Support #17762

rondogency · 2020-03-04T19:27:33Z

Description

Add random number generator support for custom operator libraries.

Design

We pass from MXNet the initialized and seeded states, located on CPU and GPU, to custom library. So user could use those seeds to generate deterministic values from a given seed passed to MXNet. Basically this workflow:

mx.random.seed(128)
r1 = mx.nd.some_custom_random_op(data)
mx.random.seed(128)
r2 = mx.nd.some_custom_random_op(data)
assert (r1 == r2)

This PR is not

Let custom library generate exactly the same sequence of random numbers comparing to MXNet

Comments

This is a continuation of the custom operator project #15921 and #17270

samskalicky · 2020-03-04T19:35:41Z

Thanks for this @rondogency! this is going to be a great addition for customOps.

A couple suggestions on the PR description:

Add random number generator support for custom operators ~~libraries~~.

Custom ~~library~~ operators now can call MXNet random number generator to get random numbers through OpResource class in forward/backward computation.

include/mxnet/lib_api.h

src/c_api/c_api.cc

eric-haibin-lin

Great. how are we testing this feature?

samskalicky · 2020-03-05T08:21:22Z

Great. how are we testing this feature?

I think @rondogency's plan is to write a dropout operator

rondogency · 2020-03-23T05:01:28Z

in the cmake, the line "target_compile_options(subgraph_lib PUBLIC -shared)" can be removed, as we added the library as shared already

wkcn

Great Work! LGTM : )

src/c_api/c_api.cc

samskalicky

Left a couple of minor items. If you dont want to do them here in this PR we can do them in #17885 instead. But while windows CI runs are blocking merging anyway, might as well just do it here ;-)

Other than that, LGTM. Thanks @rondogency for another great contribution!!!

samskalicky · 2020-03-28T06:52:42Z

@mxnet-label-bot add [pr-awaiting-merge]

wkcn · 2020-04-04T10:32:57Z

@mxnet-bot run ci [unix-gpu, centos-cpu]

mxnet-bot · 2020-04-04T10:33:06Z

Jenkins CI successfully triggered : [centos-cpu, unix-gpu]

wkcn · 2020-04-05T01:52:42Z

@mxnet-bot run ci [unix-gpu]

mxnet-bot · 2020-04-05T01:52:48Z

Jenkins CI successfully triggered : [unix-gpu]

rondogency · 2020-04-05T20:16:37Z

@mxnet-bot run ci [unix-gpu]

mxnet-bot · 2020-04-05T20:16:42Z

Jenkins CI successfully triggered : [unix-gpu]

rondogency · 2020-04-06T23:43:45Z

@mxnet-bot run ci [unix-gpu]

mxnet-bot · 2020-04-06T23:43:54Z

Jenkins CI successfully triggered : [unix-gpu]

rondogency · 2020-04-07T17:23:16Z

@mxnet-bot run ci [unix-gpu, windows-gpu]

mxnet-bot · 2020-04-07T17:23:23Z

Jenkins CI successfully triggered : [windows-gpu, unix-gpu]

Add random number generator support for custom operator libraries. Design: We pass from MXNet the initialized and seeded states, located on CPU and GPU, to custom library. So user could use those seeds to generate deterministic values from a given seed passed to MXNet. Basically this workflow: mx.random.seed(128) r1 = mx.nd.some_custom_random_op(data) mx.random.seed(128) r2 = mx.nd.some_custom_random_op(data) assert (r1 == r2) This PR does not let custom library generate exactly the same sequence of random numbers comparing to MXNet This is a continuation of the custom operator project apache#15921 and apache#17270

…18069) * Dynamic subgraph compile support (#17623) This PR adds support for passing the NDArrays from the existing optimize_for API down to the reviewSubgraph function in an external library. It also adds a new API for HybridBlock called optimize_for that can partition the model without running a forward pass. Feature changes Adds new API to HybridBlock optimize_for that partitions the model but does not call the cachedOp Modifies the subgraph library example to optionally require args to be provided Adds annotation on subgraph inputs for the name of the original param so that inputs can be mapped and passes annotations to input nodes of subgraphs Adds support for tensors in MKLDNN format, calls Reorder2Default New tests Adds a new test to partition operators that directly consume params add a new model to test where ops to be partitioned have args/params Bug Fixes fixes bug in passing ids vector by value instead of by reference fixes bug in passing copies of attributes instead of by reference fixes bug where _cached_graph was not updated after partitioning fixes memory leak where user-specified attributes on subgraph ops were not freed if subgraph was rejected fixes problem incorrectly indexing into shape/dtype maps when annotating the graph Docs Updates the README doc with the latest changes described above * Adding sparse support to MXTensor for custom operators (#17569) * Added enum for sparse storage * Add structure for Dense and Sparse * redesign the data structure for MXSparse * pull out aux data from sparse NDArray * Added more sparse arguments to API interface * Passed sparse from c_api to lib_api.h and set in MXTensor * Fix indent * fix segfault * Fix NDArray to MXTensor errors * Add a sample of sparse(CSR) transpose * Make CSR transpose temporarily work by hardcoding * Fixed sparse output size(Refined) * Add tests for symbolic and stateful ops * Added a sample for row sparse transpose * Added real row sparse transpose * Fix output size issue by adding lambda for CheckAndAlloc() * Fix mixed storage formats error * Added infer storage type function * resolve comments * Set inferSType as optional function * Resolve comments * Add error messages * Resolve comments * verify transpose ops results * fix sanity check * update MX_LIBRARY_VERSION to 5 * Custom Operator Random Number Generator Support (#17762) Add random number generator support for custom operator libraries. Design: We pass from MXNet the initialized and seeded states, located on CPU and GPU, to custom library. So user could use those seeds to generate deterministic values from a given seed passed to MXNet. Basically this workflow: mx.random.seed(128) r1 = mx.nd.some_custom_random_op(data) mx.random.seed(128) r2 = mx.nd.some_custom_random_op(data) assert (r1 == r2) This PR does not let custom library generate exactly the same sequence of random numbers comparing to MXNet This is a continuation of the custom operator project #15921 and #17270 Co-authored-by: guanxinq <[email protected]> Co-authored-by: Ziyi Mu <[email protected]>

* Dynamic subgraph compile support (#17623) This PR adds support for passing the NDArrays from the existing optimize_for API down to the reviewSubgraph function in an external library. It also adds a new API for HybridBlock called optimize_for that can partition the model without running a forward pass. Feature changes Adds new API to HybridBlock optimize_for that partitions the model but does not call the cachedOp Modifies the subgraph library example to optionally require args to be provided Adds annotation on subgraph inputs for the name of the original param so that inputs can be mapped and passes annotations to input nodes of subgraphs Adds support for tensors in MKLDNN format, calls Reorder2Default New tests Adds a new test to partition operators that directly consume params add a new model to test where ops to be partitioned have args/params Bug Fixes fixes bug in passing ids vector by value instead of by reference fixes bug in passing copies of attributes instead of by reference fixes bug where _cached_graph was not updated after partitioning fixes memory leak where user-specified attributes on subgraph ops were not freed if subgraph was rejected fixes problem incorrectly indexing into shape/dtype maps when annotating the graph Docs Updates the README doc with the latest changes described above * Adding sparse support to MXTensor for custom operators (#17569) * Added enum for sparse storage * Add structure for Dense and Sparse * redesign the data structure for MXSparse * pull out aux data from sparse NDArray * Added more sparse arguments to API interface * Passed sparse from c_api to lib_api.h and set in MXTensor * Fix indent * fix segfault * Fix NDArray to MXTensor errors * Add a sample of sparse(CSR) transpose * Make CSR transpose temporarily work by hardcoding * Fixed sparse output size(Refined) * Add tests for symbolic and stateful ops * Added a sample for row sparse transpose * Added real row sparse transpose * Fix output size issue by adding lambda for CheckAndAlloc() * Fix mixed storage formats error * Added infer storage type function * resolve comments * Set inferSType as optional function * Resolve comments * Add error messages * Resolve comments * verify transpose ops results * fix sanity check * update MX_LIBRARY_VERSION to 5 * Custom Operator Random Number Generator Support (#17762) Add random number generator support for custom operator libraries. Design: We pass from MXNet the initialized and seeded states, located on CPU and GPU, to custom library. So user could use those seeds to generate deterministic values from a given seed passed to MXNet. Basically this workflow: mx.random.seed(128) r1 = mx.nd.some_custom_random_op(data) mx.random.seed(128) r2 = mx.nd.some_custom_random_op(data) assert (r1 == r2) This PR does not let custom library generate exactly the same sequence of random numbers comparing to MXNet This is a continuation of the custom operator project #15921 and #17270 Co-authored-by: guanxinq <[email protected]> Co-authored-by: Ziyi Mu <[email protected]>

rondogency added 15 commits February 26, 2020 15:15

prototype of rng

fdb9b66

type fix

83d087c

use union type

ce8b82f

misc fix

87d3314

add stateful rng

001daeb

use enum call type and add seed

f7a8371

add gpu rng

57c877e

add uniform and normal

ec20bb3

fix switch

59cccef

fix return type

4009092

add float uniform normal

1c82570

fix float32

068a3c6

misc fix

cad0f53

fix on type

b1f1762

sanity check

e133f69

rondogency requested review from anirudh2290 and eric-haibin-lin as code owners March 4, 2020 19:27

samskalicky reviewed Mar 4, 2020

View reviewed changes

include/mxnet/lib_api.h Outdated Show resolved Hide resolved

wkcn reviewed Mar 5, 2020

View reviewed changes

src/c_api/c_api.cc Outdated Show resolved Hide resolved

eric-haibin-lin reviewed Mar 5, 2020

View reviewed changes

rondogency added 3 commits March 18, 2020 15:00

test ci

499ec88

retry ci

f835883

merge master and pump version

a127520

rondogency added 2 commits March 23, 2020 19:52

merge master after sparse

5e6cd74

new impl for passing only states

7e34d7a

rondogency requested review from aaronmarkham and szha as code owners March 26, 2020 21:42

wkcn approved these changes Mar 28, 2020

View reviewed changes

samskalicky reviewed Mar 28, 2020

View reviewed changes

src/c_api/c_api.cc Outdated Show resolved Hide resolved

samskalicky reviewed Mar 28, 2020

View reviewed changes

src/c_api/c_api.cc Outdated Show resolved Hide resolved

samskalicky approved these changes Mar 28, 2020

View reviewed changes

lanking520 added the pr-awaiting-merge Review and CI is complete. Ready to Merge label Mar 28, 2020

rondogency added 2 commits March 29, 2020 22:27

nit fix

9877b5e

Merge branch 'master' into custom_op_rng

ed4214d

rondogency added 2 commits April 6, 2020 22:11

Merge branch 'master' into custom_op_rng

2c3d3d6

fix empty vector issue

eb590ed

samskalicky mentioned this pull request Apr 7, 2020

Dynamic subgraph accesses elements of empty vector #17987

Closed

leezu merged commit 16ddc6d into apache:master Apr 8, 2020

rondogency deleted the custom_op_rng branch April 8, 2020 00:47

samskalicky mentioned this pull request Apr 15, 2020

[1.7] MXNet Extension PRs (#17623, #17569, #17762) #18063

Merged

samskalicky mentioned this pull request Apr 15, 2020

[1.7] Backport MXNet Extension PRs (#17623, #17569, #17762) #18063 #18069

Merged

eric-haibin-lin mentioned this pull request Apr 16, 2020

Row-sparse constant initializer accesses out of bound elements #17988

Open

samskalicky mentioned this pull request May 4, 2020

[RFC] Custom Operator Part 2 #17006

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Custom Operator Random Number Generator Support #17762

Custom Operator Random Number Generator Support #17762

rondogency commented Mar 4, 2020 •

edited

Loading

samskalicky commented Mar 4, 2020

eric-haibin-lin left a comment

samskalicky commented Mar 5, 2020

rondogency commented Mar 23, 2020

wkcn left a comment

samskalicky left a comment

samskalicky commented Mar 28, 2020

wkcn commented Apr 4, 2020

mxnet-bot commented Apr 4, 2020

wkcn commented Apr 5, 2020

mxnet-bot commented Apr 5, 2020

rondogency commented Apr 5, 2020

mxnet-bot commented Apr 5, 2020

rondogency commented Apr 6, 2020

mxnet-bot commented Apr 6, 2020

rondogency commented Apr 7, 2020

mxnet-bot commented Apr 7, 2020

Custom Operator Random Number Generator Support #17762

Custom Operator Random Number Generator Support #17762

Conversation

rondogency commented Mar 4, 2020 • edited Loading

Description

Design

This PR is not

Comments

samskalicky commented Mar 4, 2020

eric-haibin-lin left a comment

Choose a reason for hiding this comment

samskalicky commented Mar 5, 2020

rondogency commented Mar 23, 2020

wkcn left a comment

Choose a reason for hiding this comment

samskalicky left a comment

Choose a reason for hiding this comment

samskalicky commented Mar 28, 2020

wkcn commented Apr 4, 2020

mxnet-bot commented Apr 4, 2020

wkcn commented Apr 5, 2020

mxnet-bot commented Apr 5, 2020

rondogency commented Apr 5, 2020

mxnet-bot commented Apr 5, 2020

rondogency commented Apr 6, 2020

mxnet-bot commented Apr 6, 2020

rondogency commented Apr 7, 2020

mxnet-bot commented Apr 7, 2020

rondogency commented Mar 4, 2020 •

edited

Loading