[Relay][ADT]Static Tensor Array #5103

kevinthesun · 2020-03-19T21:58:46Z

Add static tensor array for fixed rank tensor array. With this change, we can more easily do type inference and optimization for most tensor array use cases.

Thanks for @wweic working on the base infra of StaticTensorArrayOps class.

@wweic @zhiics @yongwww @icemelon9

MarisaKirisame · 2020-03-19T23:03:40Z

There are lots of AST written by hand. Can you try using the relay parser?

masahi · 2020-03-19T23:30:52Z

Is the constraint only fixed "rank"? I have a use case for fixed "shape" tensor array (a tighter constraint)
https://discuss.tvm.ai/t/a-tensor-array-usage-question-stacking-a-list-of-fixed-sized-tensors/5565

kevinthesun · 2020-03-20T00:04:12Z

There are lots of AST written by hand. Can you try using the relay parser?

@MarisaKirisame This PR follows the patten of current Tensor Array. What do you mean by using relay parser?

kevinthesun · 2020-03-20T00:08:10Z

https://discuss.tvm.ai/t/a-tensor-array-usage-question-stacking-a-list-of-fixed-sized-tensors/5565

@masahi fixed shape is a subset of fixed rank and supported in this PR. For stack operator, even if we have all tensors in the array with the same static shape, the first axis output shape will still be Any().

masahi · 2020-03-20T00:17:32Z

@kevinthesun Great! I've been waiting for this (thanks @wweic). I think the first output axis of stack being Any makes sense and it is no problem for me, assuming it is the length of the array.

@MarisaKirisame This PR follows the patten of current Tensor Array. What do you mean by using relay parser?

I think Marisa is talking about writing the new ADT and functions in the "Relay language" itself and use the parser to make it available to python, similar to the way List ADT and its helper functions are implemented in Prelude.

kevinthesun · 2020-03-20T00:31:46Z

I think Marisa is talking about writing the new ADT and functions in the "Relay language" itself and use the parser to make it available to python, similar to the way List ADT and its helper functions are implemented in Prelude.

I see. My understanding for this is List ADT primitives are small and mature enough to be implemented in prelude. We can also move tensor array related primitives into prelude when they becomes more mature. In addition, we might want to move the generic tensor array as well, not just static tensor array. Does this make sense?

masahi · 2020-03-20T00:41:05Z

Does this make sense?

Yes. Since the generic tensor array is already implemented in this way, I also think it is better to let this in first and later port all tensor array stuff to Relay lang. There could be common code between generic/static that should be refactored in the process.

wweic · 2020-03-31T12:38:59Z

@MarisaKirisame The reason we can not use relay parser is that we want to dynamically generate the operators for specific shape while we convert the TF model to relay IR. Current relay text parser can not easily do that.

wweic · 2020-03-31T12:40:40Z

python/tvm/relay/prelude.py

+        shape_str = str(self.shape).replace('[', '').replace(']', '')\
+            .replace('(', '').replace(')', '').replace(', ', '_')\
+            .replace(',', '')


maybe we should improve this a bit. can we use '_'.join(self.shape)?

wweic · 2020-03-31T12:50:50Z

python/tvm/relay/prelude.py

+        """Defines the dynamic tensor ADT, which is the container for tensors
+        with variable shapes."""


Suggested change

"""Defines the dynamic tensor ADT, which is the container for tensors

with variable shapes."""

"""Defines the static tensor ADT, which is the container for tensors

with fixed shape."""

python/tvm/relay/prelude.py

wweic · 2020-03-31T12:56:01Z

python/tvm/relay/prelude.py

+        lower = Var('lower', scalar_type('int32'))
+        upper = Var('upper', scalar_type('int32'))
+        tvar = Var('t')
+        case = Clause(PatternConstructor(self.get_var('tensor_constructor'), [PatternVar(tvar)]),


Suggested change

case = Clause(PatternConstructor(self.get_var('tensor_constructor'), [PatternVar(tvar)]),

case = Clause(PatternConstructor(tensor_constructor, [PatternVar(tvar)]),

python/tvm/relay/prelude.py

wweic · 2020-03-31T13:00:59Z

python/tvm/relay/prelude.py

+            Function([x], Match(x, [case], False), tensor_type_var(), [])
+
+    def define_tensor_array_read(self):
+        """Defines a function to get the head of a list. Assume the list has at least one


Suggested change

"""Defines a function to get the head of a list. Assume the list has at least one

"""Defines a function to get the nth element of a list. Assume the list has at least one

wweic · 2020-03-31T13:04:13Z

python/tvm/relay/prelude.py

+            list[tensor_t] -> Tensor[(Any), int32] -> tensor_t -> list[tensor_t]
+
+        Set static indices shape by specifying indices_shape.
+        Set for_update to get static indices shape operator.


Suggested change

Set for_update to get static indices shape operator.

Set force_update to get static indices shape operator.

kevinthesun · 2020-04-01T17:49:18Z

@wweic Comments addressed. PTAL.

wweic

lgtm. some minor comments.

wweic · 2020-04-02T13:29:44Z

python/tvm/relay/prelude.py

@@ -217,7 +218,7 @@ def define_tensor_array_unstack(self):
        tensor_array_unstack_tensor(t) : tensor_t -> list[tensor_t]
        """
        ndim = len(self.shape)
-        # Skip scalar case
+        # We don't register unstask for scalar tensor array


Suggested change

# We don't register unstask for scalar tensor array

# We don't register unstack for scalar tensor array

python/tvm/relay/prelude.py

kevinthesun · 2020-04-03T17:22:16Z

@wweic Fixed.

kevinthesun · 2020-04-05T20:42:47Z

Thanks @wweic @masahi @MarisaKirisame

* Add other static tensor array ops * Add tensor array get data * Minor refactor * Fix pylint * Update docstring * Make get data more generic * Improve test * Improve split test * Improve get data * Minor fix * Further improvement for static shape * Improve shape parsing * Unify get_static_name

kevinthesun requested a review from wweic March 19, 2020 21:58

kevinthesun added the status: need review label Mar 19, 2020

kevinthesun force-pushed the static-tensor-array-ops branch from fa9e4a7 to 92d7eaf Compare March 25, 2020 00:00

tqchen assigned wweic Mar 30, 2020

kevinthesun force-pushed the static-tensor-array-ops branch from 8a4b4fc to 769cdbd Compare March 31, 2020 01:16

wweic reviewed Mar 31, 2020

View reviewed changes

wweic requested changes Apr 2, 2020

View reviewed changes

kevinthesun added 13 commits April 3, 2020 00:49

Add other static tensor array ops

79ba679

Add tensor array get data

3d208cf

Minor refactor

02ea77d

Fix pylint

0882c23

Update docstring

5ec1683

Make get data more generic

e69f4e3

Improve test

d4d18cb

Improve split test

0c4df55

Improve get data

08c8c74

Minor fix

e80c9c3

Further improvement for static shape

d22c92e

Improve shape parsing

fdb37e2

Unify get_static_name

183ac9e

kevinthesun force-pushed the static-tensor-array-ops branch from 4a9dd1d to 183ac9e Compare April 3, 2020 00:50

wweic approved these changes Apr 5, 2020

View reviewed changes

kevinthesun merged commit b5352ee into apache:master Apr 5, 2020

kevinthesun deleted the static-tensor-array-ops branch April 5, 2020 20:42

masahi mentioned this pull request Apr 11, 2020

[Torch] Support Python list, more realistic recurrent networks #5306

Merged

ZihengJiang mentioned this pull request Sep 25, 2020

TVM v0.7 Release Note Candidate #6486

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Relay][ADT]Static Tensor Array #5103

[Relay][ADT]Static Tensor Array #5103

kevinthesun commented Mar 19, 2020 •

edited

Loading

MarisaKirisame commented Mar 19, 2020

masahi commented Mar 19, 2020

kevinthesun commented Mar 20, 2020 •

edited

Loading

kevinthesun commented Mar 20, 2020 •

edited

Loading

masahi commented Mar 20, 2020

kevinthesun commented Mar 20, 2020

masahi commented Mar 20, 2020

wweic commented Mar 31, 2020

wweic Mar 31, 2020

wweic Mar 31, 2020

wweic Mar 31, 2020

wweic Mar 31, 2020

wweic Mar 31, 2020

kevinthesun commented Apr 1, 2020

wweic left a comment

wweic Apr 2, 2020

kevinthesun commented Apr 3, 2020

kevinthesun commented Apr 5, 2020

		"""Defines the dynamic tensor ADT, which is the container for tensors
		with variable shapes."""

	case = Clause(PatternConstructor(self.get_var('tensor_constructor'), [PatternVar(tvar)]),
	case = Clause(PatternConstructor(tensor_constructor, [PatternVar(tvar)]),

	"""Defines a function to get the head of a list. Assume the list has at least one
	"""Defines a function to get the nth element of a list. Assume the list has at least one

	Set for_update to get static indices shape operator.
	Set force_update to get static indices shape operator.

	# We don't register unstask for scalar tensor array
	# We don't register unstack for scalar tensor array

[Relay][ADT]Static Tensor Array #5103

[Relay][ADT]Static Tensor Array #5103

Conversation

kevinthesun commented Mar 19, 2020 • edited Loading

MarisaKirisame commented Mar 19, 2020

masahi commented Mar 19, 2020

kevinthesun commented Mar 20, 2020 • edited Loading

kevinthesun commented Mar 20, 2020 • edited Loading

masahi commented Mar 20, 2020

kevinthesun commented Mar 20, 2020

masahi commented Mar 20, 2020

wweic commented Mar 31, 2020

wweic Mar 31, 2020

Choose a reason for hiding this comment

wweic Mar 31, 2020

Choose a reason for hiding this comment

wweic Mar 31, 2020

Choose a reason for hiding this comment

wweic Mar 31, 2020

Choose a reason for hiding this comment

wweic Mar 31, 2020

Choose a reason for hiding this comment

kevinthesun commented Apr 1, 2020

wweic left a comment

Choose a reason for hiding this comment

wweic Apr 2, 2020

Choose a reason for hiding this comment

kevinthesun commented Apr 3, 2020

kevinthesun commented Apr 5, 2020

kevinthesun commented Mar 19, 2020 •

edited

Loading

kevinthesun commented Mar 20, 2020 •

edited

Loading

kevinthesun commented Mar 20, 2020 •

edited

Loading