[FRONTEND] A Python hybrid frontend #1251

were · 2018-06-08T20:23:37Z

Hybrid Frontend Developer Guide

This hybrid frontend is aimed at:

Building IR in a more intuitive way
Writing preliminary versions of some idioms that yet have not been supported by

Features

Software emulation

This feature supports both software emulation and compilation of the code.

To define a function, you need to use tvm.hybrid.script decorator to indicate this is a hybrid function:

@tvm.hybrid.script
def outer_product(a, b, c):
    for i in range(a.shape[0]):
        for j in range(b.shape[0]):
            c[i, j] = a[i] * b[j]
a = numpy.random.rand(100)
b = numpy.random.rand(99)
c = numpy.zeros((100, 99))
outer_product(a, b)

This decorator will help you to import key words required spontaneously when software emulation.
Every element in the argument list is either a python variable or numpy tensor.

Backend Compilation

The current parse interface looks like:

a = tvm.placeholder((100, ), name='a')
b = tvm.placeholder((99, ), name='b')
c = tvm.placeholder((100, 99), name='c')
tvm.hybrid.parse(outer_product, [a, b, c]) # return an ir root of this function

TODO: If we pass these tvm tensors to this function, it returns a op node:

a = tvm.placeholder((100, ), name='a')
b = tvm.placeholder((99, ), name='b')
c = tvm.placeholder((100, 99), name='c')
op = outer_product(a, b, c) # return the corresponding op node

Scheduling

Under construction, not truly supported yet.

Follow up the example above, you can use some tvm like interfaces to manipulate the structure of IR:

sch = tvm.create_schedule(op)
jo, ji = sch.split(j, 4)
sch.vectorize(ji)

split, reorder, and loop_annotation will be supported!

Attributes

So far, ONLY tensors' shape attribute is supported!

Loops

In HalideIR, loops have in total 4 types: serail, unrolled, parallel, and vectorized.

Here we use range, serial, unroll, parallel, and vectorize, these 5 keywords to annotate the types of for loops.

NOTE: In HalideIR those are enums, they are in passive form. Here we use active form to annotate loops, because they are ready to run.

NOTE: Unlike what that is in HalideIR, in loop_type(a, b), a is the starting point and b is the trip count of iterations. Here loop_type(a, b) indicates [a, b).

Variables

Because there is no variables in HalideIR, all the mutatable variables will be lowered to an array with size 1.
It takes the first store of a variable as its declaration.
NOTE: Unlike conventional Python, the declared array can only be used in the scope level it is declared.

for i in range(5):
    sum = 0
    for j in range(5):
    	sum += a[i, j] #do something with sum
    b[i] = sum #you can still use sum in this level
#you can NEVER use some here, even though it is allowed in conventional Python
a[0] = sum

Conditional Statement and Expression

if condition:
    # do something
a = b if condition else c

However, NO True and False keyword supported yet.

Math intrinsics

So far, these math intrinsics, log, exp, sigmoid, tanh, power, and popcount, are supported. No import is required, just use it!

Array allocation

TODO: Use a function call allocation(shape, type, share/local) to declare an array buffer. The basic usage is roughly the same as variables

Thread bind

You can also do loop-thread bind by writing code like this:

for tx in bind("threadIdx.x", 100):
    a[tx] = b[tx]

Appendix

Keywords

Statement keywords: for, in, if, else
For keywords: serial, range, unroll, parallel, vectorize, bind
Math keywords: log, exp, sigmoid, tanh, power, popcount

were · 2018-06-08T20:29:04Z

@tqchen @merrymercy @kevinthesun @xqdan @ZihengJiang
I think these are reviews? I hope I did not miss any one.

merrymercy · 2018-06-10T22:08:17Z

docs/dev/hybrid_script.md

+a = numpy.random.rand(100)
+b = numpy.random.rand(99)
+c = numpy.zeros((100, 99))
+outer_product(a, b)


should be outer_product(a, b, c)

merrymercy

Other parts look good to me

tqchen · 2018-06-12T00:25:43Z

@were please comment when the PR is ready to be reviewed. We shouldn't put TODO in the docs, but you can put a note saying that the feature is yet to be supported.

We will need a tutorial in tutorials/language/hybrid_script.py

liao02x

a few typos

liao02x · 2018-06-13T05:06:21Z

docs/dev/hybrid_script.md

+
+### Loops
+
+In HalideIR, loops have in total 4 types: `serail`, `unrolled`, `parallel`, and `vectorized`.


serail should be serial

liao02x · 2018-06-13T05:07:43Z

docs/dev/hybrid_script.md

+    for j in range(5):
+    	sum += a[i, j] #do something with sum
+    b[i] = sum #you can still use sum in this level
+#you can NEVER use some here, even though it is allowed in conventional Python


Is that some, or should it be sum?

tqchen · 2018-06-13T16:57:12Z

general progress tracked in #1213

tqchen · 2018-06-13T16:58:29Z

python/tvm/build_module.py

+        stmt = schedule.ScheduleOps(sch, bounds)
+        stmt = ir_pass.InjectPrefetch(stmt)
+    else:
+        #So far there is no op for hybrid script, so a plain ir body is given


check if sch is stmt, if not raise error

tqchen · 2018-06-13T17:00:54Z

python/tvm/hybrid/_internal.py

+from ..tensor import Tensor
+
+# Useful constants
+NOP = _make.Evaluate(_api.const(0, dtype='int32'))


try not to construct IR object in global, since we sometimes will have runtime only object and we need to support non-dependency when we only import but did not use hybrid.

Construct a Constant class single and overload property to do it lazily when any of them is requested

tqchen · 2018-06-13T17:01:24Z

python/tvm/hybrid/_internal.py

+    NUMPY_ARG_TYPES = (float, int, long, numpy.float32, numpy.int32, numpy.ndarray)
+
+def _is_tvm_arg_types(args):
+    """Determine a list of element is either a list of tvm arguments of a list of numpy arguments.


when possible, document the arg types

tqchen · 2018-06-13T17:01:36Z

python/tvm/hybrid/_internal.py

+
+TVM_ARG_TYPES = (_expr.Var, Tensor)
+if sys.version_info[0] == 3:
+    NUMPY_ARG_TYPES = (float, int, numpy.float32, numpy.int32, numpy.ndarray)


this should go to _ffi.base

tqchen · 2018-06-13T17:04:10Z

python/tvm/hybrid/_intrin.py

@@ -0,0 +1,69 @@
+"""Intrinsics of Python-Halide DSL for Python runtime"""


Intrinsics of TVM Hybrid script

tqchen · 2018-06-13T17:14:27Z

python/tvm/hybrid/parser.py

+        Returns
+        -------
+        func_name: str
+        The name of the function to be lowered; if not provided,


indent the description

tqchen · 2018-06-13T17:15:08Z

python/tvm/hybrid/parser.py

+
+def parse_python(src, args):
+    """ The helper function of calling the AST visitor"""
+    root = ast.parse(src)


seems most of this can be merged into parser construction

tqchen · 2018-06-13T17:15:48Z

python/tvm/hybrid/parser.py

+
+    #pylint: disable=missing-docstring, invalid-name
+    #pylint: disable=consider-merging-isinstance, no-else-return
+    #pylint: disable=inconsistent-return-statements


check inconsistent-return-statements

tqchen · 2018-06-13T17:16:09Z

python/tvm/hybrid/var_decl.py

+from ._intrin import HYBRID_GLOBALS
+
+class PyVariableUsage(ast.NodeVisitor):
+    """The vistor class to determine the declaration, r/w status, and last use of each variable"""


document arguments

tqchen · 2018-06-13T17:16:20Z

python/tvm/hybrid/var_decl.py

@@ -0,0 +1,68 @@
+"""Determines the declaration, r/w status, and last use of each variable"""


import absolute import

tqchen · 2018-06-20T17:17:53Z

docs/dev/hybrid_script.rst

@@ -0,0 +1,66 @@
+Hybrid Frontend Developer Guide
+-------------------------------


Let us start from first level title =======

tqchen · 2018-06-21T16:19:43Z

docs/langref/hybrid_script.rst

@@ -0,0 +1,169 @@
+Hybrid Frontend Language Reference
+----------------------------------


first level title is =====, second level title is -------, third level title is ~~~~~

tqchen · 2018-06-21T16:20:02Z

@Laurawly please also take a look

tqchen · 2018-06-21T16:20:59Z

docs/dev/hybrid_script.rst

+If you are a developer:
+
+1. who is trying writing some preliminary patterns that have not been supported by TVM yet,
+maybe ``lang_ref/hybrid_script.rst`` is a better place for you.


use ref feature of rst

tqchen · 2018-06-21T16:22:38Z

docs/langref/hybrid_script.rst

+
+In HalideIR, loops have in total 4 types: ``serial``, ``unrolled``, ``parallel``, and ``vectorized``.
+
+Here we use ``range``, ``serial``, ``unroll``, ``parallel``, and ``vectorize``,


remove serial

tqchen · 2018-06-21T16:23:39Z

docs/langref/hybrid_script.rst

+
+**NOTE**: In HalideIR those are enums, they are in passive form.
+          Here we use active form to annotate loops, because they are ready to run.
+


since this is useer facing doc, no need to refer back to HalideIR, instead, directly say the loop_type corresponds to range. Use note block construction of rst .

tqchen · 2018-06-21T16:23:51Z

docs/langref/hybrid_script.rst

+All the mutatable variables will be lowered to an array with size 1.
+It regards the first store of a variable as its declaration.
+
+**NOTE**: Unlike conventional Python, in hybrid script, the declared variable


use note block rst

tqchen · 2018-06-21T20:28:32Z

docs/dev/hybrid_script.rst

+If you are a developer:
+
+1. who is trying writing some preliminary patterns that have not been supported by TVM yet,
+maybe `language ref <../langref/hybrid_script.rst>`_ is a better place for you.


Use ref. see http://www.sphinx-doc.org/en/stable/markup/inline.html cross reference arbitrary locations

This is a ref to a file instead of an anchor. Can I still use label to do it? I suppose label is only for intra file instead of inter.

label works for inter file

tqchen · 2018-06-21T20:29:05Z

docs/langref/hybrid_script.rst

@@ -0,0 +1,170 @@
+Hybrid Frontend Language Reference


Hybrid Script Language Reference

since this is the first level, change to ======

tqchen · 2018-06-21T20:30:04Z

python/tvm/hybrid/_intrin.py

+
+
+def allocate(shape, dtype=None):
+    """Allocate a buffer with given shape"""


document all the arguments, see https://docs.tvm.ai/contribute/document.html#document-python

tqchen · 2018-06-21T20:30:09Z

python/tvm/hybrid/_intrin.py

+
+
+def popcount(x):
+    """Software emulated popcount function which counts 1's in a number's binary representation."""


https://docs.tvm.ai/contribute/document.html#document-python

tqchen · 2018-06-21T20:30:14Z

python/tvm/hybrid/_intrin.py

+    return cnt
+
+
+def sigmoid(x):


https://docs.tvm.ai/contribute/document.html#document-python

tqchen · 2018-06-21T20:31:07Z

python/tvm/hybrid/_util.py

@@ -0,0 +1,73 @@
+"""Internal utilities for parsing Python subset to HalideIR"""


consider just make it util.py

tqchen · 2018-06-21T20:31:21Z

python/tvm/hybrid/_util.py

+
+
+# Useful constants. In avoid of runtime dependences, we use function calls to return them.
+def make_nop():


document functions

tqchen · 2018-06-21T20:32:43Z

python/tvm/hybrid/api.py

+    -------
+    (halide_ir, parser) : (Stmt, PyAST2HalideIR)
+        The result Halide IR and the parser class instance.
+        TODO: Later we deprecate this return value, use a dedicated OP node type instead


parse should always return Stmt only

tqchen · 2018-06-21T20:33:04Z

python/tvm/hybrid/parser.py

@@ -0,0 +1,342 @@
+"""Compiling a TVM Hybrid Script Python to HalideIR"""


Hybrid Script Parser

tqchen · 2018-06-21T20:34:18Z

python/tvm/hybrid/parser.py

+        return res
+
+
+def parse_python(src, args):


is this function necessary? Can we just do and inline it into parse?

HybridParser parser(src, args) return parser.parse(root)

I just want to make the code style uniform, each IR pass has its own helper function. Just like that is in Halide style.

tqchen · 2018-06-21T20:35:58Z

python/tvm/hybrid/__init__.py

@@ -0,0 +1,10 @@
+"""Hybrid Programming APIs of TVM Python Package.
+


need to add hybrid.rst to the python API, see example here
https://github.com/dmlc/tvm/blob/master/docs/api/python/index.rst
https://github.com/dmlc/tvm/blob/master/docs/api/python/intrin.rst

Laurawly · 2018-06-21T21:07:31Z

@were Could there be more gpu testings regarding vthread injection and usage of shared / warp memory?

tqchen · 2018-06-22T20:48:24Z

Thanks! This is merged. Please followup with PRs on gpu support

were mentioned this pull request Jun 8, 2018

[FRONTEND] Hybrid python frontend #1213

Closed

17 tasks

were changed the title ~~cleanup a branch I messed up~~ [FRONTEND] A Python hybrid frontend Jun 8, 2018

merrymercy reviewed Jun 10, 2018

View reviewed changes

merrymercy requested changes Jun 10, 2018

View reviewed changes

tqchen added the status: review in progress label Jun 12, 2018

liao02x approved these changes Jun 13, 2018

View reviewed changes

tqchen requested changes Jun 13, 2018

View reviewed changes

tqchen added the status: need update need update based on feedbacks label Jun 14, 2018

Jian Weng added 19 commits June 18, 2018 12:36

cleanup a branch I messed up

1e65ab8

cleanup unused import

9578c2a

gpu skip?

c37eabc

Fix typo in developer tutorial

60ba428

Erase TODO, typo fixed.

9b70244

move np_type to _ffi.base; get rid of constant ir nodes

6227166

move all the assertions to ValueError; adjust the test case!

ab523d7

fix lint

955c0be

commit before rebase

c3f4db3

preview of rst

c1d48d2

preview code block

5049683

dont be too stingy to put blank lines!

382c751

preview subsubsections

bfadeb3

keywords label?

7d6bb8d

keywords label??

d2a01ff

anchor ref!

7b76c87

no label required?

f72eb42

no anchor!

7bb75a9

inline ZERO to fit online tests?

a8b8528

Jian Weng added 2 commits June 19, 2018 11:53

hybrid script dev guide add!

09dd994

I think I need a rst IDE

9c3c545

tqchen reviewed Jun 21, 2018

View reviewed changes

tqchen requested changes Jun 21, 2018

View reviewed changes

Jian Weng added 6 commits June 21, 2018 12:15

update the docs

742ffd3

note!

f40cf57

link?

799f1d3

link!

9196947

no underscore!

42d5615

inner anchor

d668188

tqchen requested changes Jun 21, 2018

View reviewed changes

Jian Weng added 4 commits June 21, 2018 14:10

follow up tianqis suggestion

41cc88c

erase serail

81b0b32

use :ref:

3e6e39c

move _xx to xx; fix lint

45a2d4c

tqchen approved these changes Jun 22, 2018

View reviewed changes

tqchen merged commit 290226e into apache:master Jun 22, 2018

tqchen added status: accepted and removed status: need update need update based on feedbacks status: review in progress labels Jun 22, 2018

were deleted the pyfrontend branch June 26, 2018 01:06

tqchen pushed a commit to tqchen/tvm that referenced this pull request Jul 6, 2018

[FRONTEND] A Python hybrid frontend (apache#1251)

5735ab0

mnuyens pushed a commit to mnuyens/tvm that referenced this pull request Jul 10, 2018

[FRONTEND] A Python hybrid frontend (apache#1251)

90db723

sergei-mironov pushed a commit to sergei-mironov/tvm that referenced this pull request Aug 8, 2018

[FRONTEND] A Python hybrid frontend (apache#1251)

e11a581


		### Loops

		In HalideIR, loops have in total 4 types: `serail`, `unrolled`, `parallel`, and `vectorized`.

		@@ -0,0 +1,69 @@
		"""Intrinsics of Python-Halide DSL for Python runtime"""

		@@ -0,0 +1,68 @@
		"""Determines the declaration, r/w status, and last use of each variable"""

		@@ -0,0 +1,66 @@
		Hybrid Frontend Developer Guide
		-------------------------------

		@@ -0,0 +1,169 @@
		Hybrid Frontend Language Reference
		----------------------------------


		In HalideIR, loops have in total 4 types: ``serial``, ``unrolled``, ``parallel``, and ``vectorized``.

		Here we use ``range``, ``serial``, ``unroll``, ``parallel``, and ``vectorize``,


		NOTE: In HalideIR those are enums, they are in passive form.
		Here we use active form to annotate loops, because they are ready to run.



		def allocate(shape, dtype=None):
		"""Allocate a buffer with given shape"""



		def popcount(x):
		"""Software emulated popcount function which counts 1's in a number's binary representation."""

		@@ -0,0 +1,73 @@
		"""Internal utilities for parsing Python subset to HalideIR"""



		# Useful constants. In avoid of runtime dependences, we use function calls to return them.
		def make_nop():

		@@ -0,0 +1,342 @@
		"""Compiling a TVM Hybrid Script Python to HalideIR"""

		@@ -0,0 +1,10 @@
		"""Hybrid Programming APIs of TVM Python Package.

[FRONTEND] A Python hybrid frontend #1251

[FRONTEND] A Python hybrid frontend #1251

Conversation

were commented Jun 8, 2018

Hybrid Frontend Developer Guide

Features

Software emulation

Backend Compilation

Scheduling

Attributes

Loops

Variables

Conditional Statement and Expression

Math intrinsics

Array allocation

Thread bind

Appendix

were commented Jun 8, 2018

merrymercy Jun 10, 2018 • edited Loading

Choose a reason for hiding this comment

merrymercy left a comment

Choose a reason for hiding this comment

tqchen commented Jun 12, 2018

liao02x left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tqchen commented Jun 13, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tqchen commented Jun 21, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Laurawly commented Jun 21, 2018

tqchen commented Jun 22, 2018

merrymercy Jun 10, 2018 •

edited

Loading