[LANG] Support for Tuple Inputs of Reducer and ComputeOp #175

ZihengJiang · 2017-06-08T22:47:41Z

tqchen · 2017-06-08T23:19:21Z

include/tvm/expr.h

@@ -43,6 +43,16 @@ using Halide::Internal::is_no_op;
 using Halide::likely;
 using Halide::likely_if_innermost;

+/*! \brief whether two array have the same content */
+template<typename T>
+bool IsSame(const Array<T>& a, const Array<T>& b) {


SameContent

tqchen · 2017-06-08T23:20:31Z

src/op/compute_op.cc

-    n->reduce_axis = n->body.as<ir::Reduce>()->axis;
+  if (n->body[0]->is_type<ir::Reduce>()) {
+    // batch reduction should have the same axis
+    n->reduce_axis = n->body[0].as<ir::Reduce>()->axis;


Add verification check here

Raise error if that is not true

tqchen · 2017-06-08T23:50:49Z

src/op/compute_op.cc

  return ret;
 }

 Operation ComputeOpNode::ReplaceInputs(
    const Operation& self,
    const std::unordered_map<Tensor, Tensor>& rmap) const {
  CHECK_EQ(self.operator->(), this);
-  Expr new_body = op::ReplaceTensor(this->body, rmap);
-  if (!new_body.same_as(this->body)) {
+  Array<Expr> new_body = ReplaceTensor(this->body, rmap);


Try to do it simply in a for loop, check the way we do it ir mutator

Maybe we can add a Array<NodeRef> UpdateArray(Array<NodeRef> arr, std::function<NodeRef(NodeRef)> fupdate) function?

tqchen · 2017-06-08T23:53:44Z

src/op/compute_op.cc

@@ -298,15 +332,26 @@ Stmt ComputeOpNode::BuildProvide(
  CHECK_EQ(stage->op.operator->(), this);

  if (IsCrossThreadReduction(this, stage)) {
+    LOG(INFO) << stage;
    // specially handle cross thread reduction.


Remove this

tqchen · 2017-06-10T02:30:36Z

src/pass/ir_util.h

@@ -12,6 +12,23 @@
 namespace tvm {
 namespace ir {

+template<typename T>
+inline Array<T> UpdateArray(Array<T> arr, std::function<T(T)> fupdate) {


simply use F fupdate, use a template argument for fupdate

tqchen · 2017-06-10T02:30:59Z

src/pass/ir_mutator.cc

-  } else {
-    return Array<Expr>(new_arr);
-  }
+  std::function<Expr(Expr)> fupdate = [m] (Expr e) { return m->Mutate(e); };


auto fupdate, std::function and lambda are different, lambda is more specialized and can trigger inline

…ifferent types

tqchen · 2017-06-10T21:19:35Z

src/op/compute_op.cc

  if (!is_one(reduce->condition)) {
-    *provide = IfThenElse::make(reduce->condition, *provide);
+    for (size_t i = 0; i < size; ++i) {
+      provides->at(i) = IfThenElse::make(reduce->condition, provides->at(i));


can we have one common condition for all the bodies?

tqchen · 2017-06-10T21:24:38Z

src/op/compute_op.cc

+  size_t size = self->body.size();
+  CHECK_GT(size, 0);
+  std::vector<const Reduce*> reduces(size);
+  for (size_t i = 0; i < size; ++i) {


if we assume common reduce, vector is not necessary?

reduces is used for type

tqchen · 2017-06-10T21:25:35Z

src/pass/ir_util.h

+ * \brief update array with an unary function
+ * \param arr array
+ * \param fupdate an unary function
+ * \return if update happens, return the new array, else return the


add \tparam to document the template argument

tqchen · 2017-06-10T21:26:31Z

src/pass/lower_thread_allreduce.cc

    }

    std::unordered_set<const Variable*> reduce_set;
-    for (size_t i = 2; i < call->args.size(); ++i) {
+    for (size_t i = 2+2*size; i < call->args.size(); ++i) {


space between operators

tqchen · 2017-06-10T21:28:56Z

src/schedule/graph.cc

@@ -321,11 +323,14 @@ Map<IterVar, Expr> ScanFixPointAnalysis(const Operation& scan_op) {
        }
      }
    } else if (op.as<ComputeOpNode>()) {
-      std::unordered_map<const Node*, TensorDimKey> vmap;
+      std::unordered_map<const Node*, std::vector<TensorDimKey>> vmap;


for compatiblity of older compiler

tqchen · 2017-06-10T21:35:34Z

src/schedule/schedule_dataflow_rewrite.cc

  reduce_stage->leaf_iter_vars = reduce_stage->all_iter_vars;
  reduce_stage->relations = Array<IterVarRelation>();
-  return factor_tensor;
+  return factor_tensors[0];


We need to return array of Exprs?

tqchen · 2017-06-10T21:58:12Z

include/tvm/ir.h

                   Array<IterVar> rdom,
-                   Expr condition = const_true());
+                   Expr condition = const_true(),
+                   int value_index = 0);


remove the default value, to be safe

never mind, forget this comment

tqchen · 2017-06-11T00:34:30Z

python/tvm/api.py

+            where = convert(True)
+        if size == 1:
+            return _make.Reduce(combiner, expr, axis, where, 0)
+        return [_make.Reduce(combiner, expr, axis, where, i)


change to tuple

tqchen · 2017-06-11T00:34:54Z

python/tvm/schedule.py

@@ -193,10 +193,13 @@ def rfactor(self, tensor, axis):

        Returns
        -------
-        tfactor : Tensor
+        tfactor : Tensor or Array<Tensor>


list of Tensor

tqchen · 2017-06-11T00:37:09Z

src/lang/ir.cc

@@ -79,11 +88,12 @@ Expr Reduce::make(CommReducer combiner, Expr source,
  for (size_t i = 0; i < axis.size(); ++i) {
    CHECK(axis[i].defined());
  }
-  n->type = source.type();
+  n->type = source[value_index].type();


if argument is passing by value, do std::move to save copy constructor

tqchen · 2017-06-11T00:38:24Z

src/op/compute_op.cc

+  return outputs;
+}
+
+bool CheckReduce(const ir::Reduce* a, const ir::Reduce* b) {


ReduceEqual

tqchen · 2017-06-11T00:41:13Z

src/op/compute_op.cc

  Array<Expr> freduce_args;
-  freduce_args.push_back(reduce->source);
+  freduce_args.push_back(make_const(UInt(32), size));


let us update comment in the intrinsic def to clarify the new convention

tqchen · 2017-06-11T00:42:30Z

src/op/compute_op.cc

-  body = AttrStmt::make(
-      res_handle, attr::storage_scope, StringImm::make("local"), body);
+  Stmt body = Block::make(reduce_body, assign_body);
+  for (int idx = size - 1; idx >= 0; --idx) {


keep revserse iteration style consistent

for (i = size; i !=0; --i)

This avoid certain case when reverse iteration is unsigned

tqchen · 2017-06-11T00:44:16Z

src/pass/lower_thread_allreduce.cc

-                        Type type,
-                        Var shared_buf,
+                        const std::vector<Type>& types,
+                        Array<Var> shared_bufs,


if we don't copy the value, pass by const ref

tqchen · 2017-06-11T00:46:24Z

python/tvm/schedule.py

            The created factored tensor.
        """
-        return _api_internal._ScheduleRFactor(self, tensor, axis)
+        factored = _api_internal._ScheduleRFactor(self, tensor, axis)
+        if len(factored) == 1:


return factored[0] if len(factored) == 1 else factored

* fix for composed symbol * fix * clean up * fix exception type

…e#14523) (apache#175) This PR enhances CanProve to handle symbolic bound. Such analysis is essential to eliminate predicates in dynamic shape workloads. We also the int set analysis singlepoint check to avoid recursion and improve the overall analysis speed. Added CanProveSinglePoint to serve previous stronger checks. The new CanProve comes with additinal strength argument that can only be used in top-level setting with stronger analysis. Added comment for future implementation efficiency. Testcases are added to cover the cases. Co-authored-by: Tianqi Chen <[email protected]>

…ache#175) Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 3 to 4.1.7. - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](actions/download-artifact@v3...v4.1.7) --- updated-dependencies: - dependency-name: actions/download-artifact dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

ZihengJiang added 4 commits June 6, 2017 10:41

Support for batch ComputeOp

5cf3323

Support for batch ComputeOp

c75d50f

Fix CrossThreadReduction

8a22418

Fix lint

efa79fb

tqchen reviewed Jun 9, 2017

View reviewed changes

ZihengJiang added 2 commits June 8, 2017 23:08

Add UpdateArray, remove support for batch reduce

8869899

Tuple input support for reduce

6703850

tqchen reviewed Jun 10, 2017

View reviewed changes

ZihengJiang added 2 commits June 10, 2017 13:02

rfactor works with multiple reducer; support multiple reducers with d…

50d20be

…ifferent types

Small fix

371a6c0

ZihengJiang changed the title ~~Support for batch ComputeOp~~ Support for Tuple Inputs of Reducer and ComputeOp Jun 10, 2017

ZihengJiang added 2 commits June 10, 2017 13:57

Small fix

7b3ba61

Merge branch 'master' of https://github.com/dmlc/tvm into dev

2da9fd9

tqchen requested changes Jun 10, 2017

View reviewed changes

tqchen reviewed Jun 10, 2017

View reviewed changes

ZihengJiang added 2 commits June 10, 2017 17:27

Change return type of rfactor to Array<Expr>

0fa0452

Fix lint

3739cb7

tqchen reviewed Jun 11, 2017

View reviewed changes

ZihengJiang added 3 commits June 10, 2017 18:39

Improve

0e7258e

Add tutorial

2c42e95

Improve tutorial

8491288

tqchen approved these changes Jun 11, 2017

View reviewed changes

tqchen mentioned this pull request Jun 11, 2017

Tuple Inputs&Outputs Support for ComputeOp #173

Closed

2 tasks

Improve tutorial

a3c596b

ZihengJiang merged commit f467f66 into apache:master Jun 11, 2017

ZihengJiang deleted the dev branch June 12, 2017 04:36

ZihengJiang changed the title ~~Support for Tuple Inputs of Reducer and ComputeOp~~ [LANG] Support for Tuple Inputs of Reducer and ComputeOp Jun 16, 2017

tqchen pushed a commit to tqchen/tvm that referenced this pull request May 26, 2018

[FRONTEND] Composed operators (apache#175)

2cdc0d4

* fix for composed symbol * fix * clean up * fix exception type

tqchen pushed a commit that referenced this pull request May 29, 2018

[FRONTEND] Composed operators (#175)

c9addb3

* fix for composed symbol * fix * clean up * fix exception type

tqchen pushed a commit to tqchen/tvm that referenced this pull request Jul 6, 2018

[FRONTEND] Composed operators (apache#175)

9a6feca

* fix for composed symbol * fix * clean up * fix exception type

sergei-mironov pushed a commit to sergei-mironov/tvm that referenced this pull request Aug 8, 2018

[FRONTEND] Composed operators (apache#175)

73bff7b

* fix for composed symbol * fix * clean up * fix exception type

vinx13 pushed a commit to vinx13/tvm that referenced this pull request Mar 9, 2022

[MetaSchedule] Vectorized Loading in Cooperative Fetching (apache#175)

dff8c72

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[LANG] Support for Tuple Inputs of Reducer and ComputeOp #175

[LANG] Support for Tuple Inputs of Reducer and ComputeOp #175

ZihengJiang commented Jun 8, 2017 •

edited

Loading

tqchen Jun 8, 2017

tqchen Jun 8, 2017

tqchen Jun 8, 2017

tqchen Jun 8, 2017

ZihengJiang Jun 9, 2017

tqchen Jun 8, 2017

tqchen Jun 10, 2017

tqchen Jun 10, 2017

tqchen Jun 10, 2017

tqchen Jun 10, 2017

ZihengJiang Jun 10, 2017

tqchen Jun 10, 2017

tqchen Jun 10, 2017

tqchen Jun 10, 2017

tqchen Jun 10, 2017

tqchen Jun 10, 2017

tqchen Jun 10, 2017

tqchen Jun 11, 2017

tqchen Jun 11, 2017

tqchen Jun 11, 2017

tqchen Jun 11, 2017

tqchen Jun 11, 2017

tqchen Jun 11, 2017

tqchen Jun 11, 2017

tqchen Jun 11, 2017

[LANG] Support for Tuple Inputs of Reducer and ComputeOp #175

[LANG] Support for Tuple Inputs of Reducer and ComputeOp #175

Conversation

ZihengJiang commented Jun 8, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ZihengJiang commented Jun 8, 2017 •

edited

Loading