[RELAY] Re-wrote the Graph Partitioner to support multiple outputs #5143

manupak · 2020-03-24T17:43:55Z

This PR aims to include support for multiple outputs in the regions that get outlined as functions for different compiler backends in the existing Partition Graph Pass. Such regions are annotated and bounded by compiler_begin and compiler_end annotation ops.

This is a required step as step 4 in RFC - BYOC. Moreover, this feature is requested in prior discussions such as Improved graph partitioning algorithm

This PR uses the utility functions provided by the AnnotationRegionSet PR.

manupak · 2020-03-24T17:46:21Z

cc : @zhiics @comaniac @tqchen

zhiics · 2020-03-25T16:01:44Z

src/relay/transforms/partition_graph.cc

-  /*! \brief Nodes in this subgraph. */
-  std::unordered_set<Expr, ObjectHash, ObjectEqual> nodes;
-};
+static const Op &compiler_begin_op = Op::Get("annotation.compiler_begin");


keep the original code style: static const Op&

Same for all the other places in the rest of PR

Fixed. Prolly feedback this to lint?

There are still many places where the style is violated. I recommend running clang-format from your editor. We have our own .clang-format file at the root dir.

@masahi thanks for tip! fixed this in this follow-up PR

zhiics · 2020-03-25T16:05:23Z

src/relay/transforms/partition_graph.cc

+ *
+ * Expected Usecase :
+ *      This pass is intended to run as the last pass in a series of passes as follows :
+ *      1) Annotate Supported Single Ops - annotated each single op with supported backends.


and 2) are inaccurate, we don't have these two annotations

Removed the use-case segment. This is not needed now; as things are still in discussion as how this pass would potentially fit in the bigger picture.

zhiics · 2020-03-25T16:09:55Z

src/relay/transforms/partition_graph.cc

+
+        if (region->GetOutputs().size() == 1) {
+          // If there is only a single output; no need to add a tuplegetitem node
+          return Call(glob_func, param_expr);


no need to create another call, just return ret

zhiics · 2020-03-25T16:11:51Z

src/relay/transforms/partition_graph.cc

-  int subgraph_id_{0};
-  std::unordered_set<std::shared_ptr<Subgraph>> subgraphs_;
+  /*!
+  * \brief Get the region an expression belongs to


align "*", same to the other functions below.

still not aligned, you want something like:

/*! * \brief */

Same to the other places as well

Hopefully done!

zhiics · 2020-03-25T16:13:22Z

tests/python/relay/test_pass_partition_graph.py

@@ -678,6 +678,81 @@ def expected():
    check_result(mod, {"y": y_data}, (8, 8), np.log(np_add))


+def test_multiple_outputs():


Can you add a test case for something like conv2d + batch_norm?

Not sure that I get you; Can you elaborate more as to how multiple outputs in regions are exercised in conv2d + batch_norm?

batch_norm has 3 outputs.

But isnt it returned as a tuple already ? Looks like a single output to me. Unless, you do some procesing afterwards in the region. Is that what you imply?

Oh yes. The case in my mind is conv2d->BN->relu. In this case, the region output would be a new tuple of the rest 2 BN outputs and 1 relu output (relu could be any op tho).

Added the testcase

zhiics · 2020-03-26T17:33:04Z

src/relay/transforms/partition_graph.cc

+    return BaseFunc(nullptr);
+  }
+
+  /*!


Hopefully done!

zhiics · 2020-03-26T17:33:09Z

src/relay/transforms/partition_graph.cc

+    return AnnotatedRegion(nullptr);
+  }
+
+  /*!


Hopefully done!

zhiics · 2020-03-26T17:34:09Z

src/relay/transforms/partition_graph.cc

-  int subgraph_id_{0};
-  std::unordered_set<std::shared_ptr<Subgraph>> subgraphs_;
+  /*!
+  * \brief Get the region an expression belongs to


still not aligned, you want something like:

/*! * \brief */

Same to the other places as well

zhiics · 2020-03-26T17:50:52Z

tests/python/relay/test_pass_partition_graph.py

+
+        return mod
+
+    # print(create_merged_graph())


remove the debugging stmt as well

Input : A Relay module that have functions with disjoint annotated regions using compiler_begin and compiler_end. There could be multiple outputs. Output : A Relay module with global functions for such disjoint annotated regions with calls inserted at the respective location Dependencies : AnnotatedRegionSet Utility class. Methodology : 1) The AnnotatedRegionSet utility class is able to construct a collection of nodes that are bound by a give annotation -- here we use compiler_begin and compiler_end 2) Initially, for each function in the module AnnotatedRegionSets are populated. 3) Then, Vistor pass is traversed until a compiler_end node is encountered that belongs to a "region". 4) When the first compiler_end of a given annotated region is found, a function is formed and inserted. a) if the region has multiple outputs, a Tuple node (capturing all outputs) is returned. 5) Thereafter, if we encounter an another output of the same annotated region, it is important to note that the function is already formed. Therefore, it will lookup the function and add a TupleGetItemNode is inserted. a) We will use the location index of "rets" of each "Region" of AnnotatedRegionSet as TupleGetItemNode index. 6) Therefore, functions will be created for all annotated regions. The name for each global function is created using "Region" id and the compiler name. Change-Id: I1372f02a845b6d3da03b561763e03a378dca263c

*removed the expected use-case as we are taking broken-down PR approach *code style fixes *some trivial one liners

*fixed an implicit copy to a move

*code style changes for comments *renamed test case multiple outputs --> mixed single multiple outputs Since the existing test case checks for both single and multiple output scenarios *added a new test case with conv2d + batch_norm *some var name changes in the test

*rebased

zhiics

LGTM

comaniac

LGTM

zhiics · 2020-03-31T18:27:16Z

Thanks @manupa-arm @comaniac

…pache#5143) * [RELAY] Re-wrote the Graph Partitioner to support multiple outputs Input : A Relay module that have functions with disjoint annotated regions using compiler_begin and compiler_end. There could be multiple outputs. Output : A Relay module with global functions for such disjoint annotated regions with calls inserted at the respective location Dependencies : AnnotatedRegionSet Utility class. Methodology : 1) The AnnotatedRegionSet utility class is able to construct a collection of nodes that are bound by a give annotation -- here we use compiler_begin and compiler_end 2) Initially, for each function in the module AnnotatedRegionSets are populated. 3) Then, Vistor pass is traversed until a compiler_end node is encountered that belongs to a "region". 4) When the first compiler_end of a given annotated region is found, a function is formed and inserted. a) if the region has multiple outputs, a Tuple node (capturing all outputs) is returned. 5) Thereafter, if we encounter an another output of the same annotated region, it is important to note that the function is already formed. Therefore, it will lookup the function and add a TupleGetItemNode is inserted. a) We will use the location index of "rets" of each "Region" of AnnotatedRegionSet as TupleGetItemNode index. 6) Therefore, functions will be created for all annotated regions. The name for each global function is created using "Region" id and the compiler name. Change-Id: I1372f02a845b6d3da03b561763e03a378dca263c * [RELAY] Re-wrote the Graph Partitioner to support multiple outputs *removed the expected use-case as we are taking broken-down PR approach *code style fixes *some trivial one liners * [RELAY] Re-wrote the Graph Partitioner to support multiple outputs *fixed an implicit copy to a move * [RELAY] Re-wrote the Graph Partitioner to support multiple outputs *code style changes for comments *renamed test case multiple outputs --> mixed single multiple outputs Since the existing test case checks for both single and multiple output scenarios *added a new test case with conv2d + batch_norm *some var name changes in the test * [RELAY] Re-wrote the Graph Partitioner to support multiple outputs *rebased

manupak mentioned this pull request Mar 25, 2020

[RELAY] Added a AnnotatedRegion utility class #5030

Merged

mbaret mentioned this pull request Mar 25, 2020

[RELAY] Add MergeCompilerRegions pass #5134

Merged

zhiics reviewed Mar 25, 2020

View reviewed changes

manupak force-pushed the partition_graph_multiout branch from 5cc9590 to a4653fd Compare March 26, 2020 12:02

zhiics reviewed Mar 26, 2020

View reviewed changes

manupak force-pushed the partition_graph_multiout branch from 99a02f9 to beda3bc Compare March 27, 2020 14:07

manupak added 5 commits March 31, 2020 16:48

[RELAY] Re-wrote the Graph Partitioner to support multiple outputs

259ec29

*removed the expected use-case as we are taking broken-down PR approach *code style fixes *some trivial one liners

[RELAY] Re-wrote the Graph Partitioner to support multiple outputs

911c421

*fixed an implicit copy to a move

[RELAY] Re-wrote the Graph Partitioner to support multiple outputs

ed2415f

*rebased

manupak force-pushed the partition_graph_multiout branch from beda3bc to ed2415f Compare March 31, 2020 15:51

zhiics approved these changes Mar 31, 2020

View reviewed changes

comaniac approved these changes Mar 31, 2020

View reviewed changes

zhiics merged commit 14ae3a6 into apache:master Mar 31, 2020

manupak mentioned this pull request Apr 1, 2020

[RELAY] Partition graph codestyle fixes #5202

Merged

ZihengJiang mentioned this pull request Sep 25, 2020

TVM v0.7 Release Note Candidate #6486

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RELAY] Re-wrote the Graph Partitioner to support multiple outputs #5143

[RELAY] Re-wrote the Graph Partitioner to support multiple outputs #5143

manupak commented Mar 24, 2020

manupak commented Mar 24, 2020

zhiics Mar 25, 2020

manupak Mar 26, 2020

masahi Apr 1, 2020 •

edited

Loading

manupak Apr 1, 2020

zhiics Mar 25, 2020

manupak Mar 26, 2020

zhiics Mar 25, 2020

manupak Mar 26, 2020

zhiics Mar 25, 2020

manupak Mar 26, 2020

zhiics Mar 26, 2020

manupak Mar 27, 2020

zhiics Mar 25, 2020

manupak Mar 26, 2020 •

edited

Loading

comaniac Mar 26, 2020

manupak Mar 26, 2020

comaniac Mar 26, 2020 •

edited

Loading

manupak Mar 27, 2020

zhiics Mar 26, 2020

manupak Mar 27, 2020

zhiics Mar 26, 2020

manupak Mar 27, 2020

zhiics Mar 26, 2020

zhiics Mar 26, 2020

manupak Mar 27, 2020

zhiics left a comment

comaniac left a comment

zhiics commented Mar 31, 2020

		@@ -678,6 +678,81 @@ def expected():
		check_result(mod, {"y": y_data}, (8, 8), np.log(np_add))


		def test_multiple_outputs():

[RELAY] Re-wrote the Graph Partitioner to support multiple outputs #5143

[RELAY] Re-wrote the Graph Partitioner to support multiple outputs #5143

Conversation

manupak commented Mar 24, 2020

manupak commented Mar 24, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

masahi Apr 1, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

manupak Mar 26, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

comaniac Mar 26, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhiics left a comment

Choose a reason for hiding this comment

comaniac left a comment

Choose a reason for hiding this comment

zhiics commented Mar 31, 2020

masahi Apr 1, 2020 •

edited

Loading

manupak Mar 26, 2020 •

edited

Loading

comaniac Mar 26, 2020 •

edited

Loading