[RELAY]Frontend darknet #2773

siju-samuel · 2019-03-11T09:20:59Z

#2246
@srkreddy1238 @zhreshold @jroesch @yzhliu @kazum Please help to review the darknet frontend.

Thanks for contributing to TVM! Please refer to guideline https://docs.tvm.ai/contribute/ for useful information and tips. After the pull request is submitted, please request code reviews from Reviewers.

jroesch · 2019-03-11T22:59:40Z

Overall looks good to me, will try to take a deeper dive later.

kazum · 2019-03-12T14:29:54Z

tests/python/frontend/darknet/test_forward.py

+    test_forward_upsample()
+    test_forward_l2normalize()
+    test_forward_activation_logistic()
+    test_forward_elu()


Please update tests/scripts/task_python_frontend.sh too so that CI can run this test file.

jroesch · 2019-03-17T02:55:06Z

@kazum can you give explicit approval if it looks good to you?

jroesch · 2019-03-17T02:57:03Z

tests/python/frontend/darknet/test_forward.py

+        urllib.urlretrieve(url, path)
+
+DARKNET_LIB = 'libdarknet2.0.so'
+DARKNETLIB_URL = 'https://github.com/siju-samuel/darknet/blob/master/lib/' \


Can we put this in a centrally located place? there is a TVM data repo somewhere, can we download from Joe's repo?

kazum · 2019-03-18T01:58:27Z

python/tvm/relay/frontend/darknet.py

+
+        else:
+            err = "Darknet layer type {} is not supported in relay.".format(layer_type)
+            raise NotImplementedError(err)


DECONVOLUTIONAL and BATCHNORM are not handled. Tests for those layers are also necessary.

None of the available darknet cfg uses deconvolutional or batchnorm as independant op. And darknet lib donot support resize_layer for this 2 ops, so we cannot validate the testcases also for this. Currently the code is removed and can be added if any new model support this 2 ops.

kazum · 2019-03-18T02:09:35Z

python/tvm/relay/frontend/darknet.py

+        processed = False
+
+        layer_type = LAYERTYPE(layer.type)
+        if LAYERTYPE.RNN == layer_type:


Is RNN supported in this PR? If yes, please add tests for that.

kazum · 2019-03-18T02:14:09Z

tests/python/frontend/darknet/test_forward.py

+from tvm.contrib import graph_runtime
+from nnvm import frontend
+from nnvm.testing.darknet import LAYERTYPE
+from nnvm.testing.darknet import __darknetffi__


I think nnvm.testing.darknet should be moved to tvm.relay.testing.darknet like the tensorflow frontend.

kazum · 2019-03-18T02:15:12Z

tests/python/frontend/darknet/test_forward.py

+from nnvm import frontend
+from nnvm.testing.darknet import LAYERTYPE
+from nnvm.testing.darknet import __darknetffi__
+import nnvm.compiler


nnvm.frontend and nnvm.compiler are not necessary.

kazum · 2019-03-18T02:24:38Z

tests/python/frontend/darknet/test_forward.py

+    '''test elu activation layer'''
+    net = LIB.make_network(1)
+    layer_1 = LIB.make_convolutional_layer(1, 224, 224, 3, 32, 1, 3, 2, 0, 1, 0, 0, 0, 0)
+    layer_1.activation = 8


8 should be ACTIVATION.ELU here. Can you replace all the magic numbers for activation types with ACTIVATION.*?

kazum · 2019-03-18T02:26:43Z

tutorials/frontend/from_darknet.py

+import sys
+import numpy as np
+import matplotlib.pyplot as plt
+from ctypes import *


sys and ctypes look not necessary.

yzhliu · 2019-03-29T02:22:13Z

@siju-samuel would you follow up the comments?

kazum · 2019-03-30T08:07:37Z

python/tvm/relay/frontend/darknet.py

+        split_res2 = split_res[2]
+    if softmax:
+        split_res3 = get_relay_op('softmax')(split_res[3], axis=2)
+    out = get_relay_op('concatenate')((split_res0, split_res[1], split_res2, split_res3), axis=2)


split_res3 is not defined when softmax is False.

siju-samuel · 2019-04-25T09:13:36Z

@kazum Thanks for the review, I have updated the comments. Could u please help to review this again. TIA

cbalint13 · 2019-04-25T19:11:39Z

python/tvm/relay/frontend/darknet.py

+    extra_pad_size = attrs.get('extra_pad_size', 0)
+    if extra_pad_size:
+        pad_width = ((0, 0), (0, 0), (0, extra_pad_size), (0, extra_pad_size))
+        inputs = get_relay_op('pad')(*inputs,


Will throw an error on yolov3-tiny (issue #3057)

tvm/relay/frontend/darknet.py", line 66, in _darknet_maxpooling return get_relay_op('max_pool2d')(*inputs, **new_attrs) TypeError: max_pool2d() argument after * must be an iterable, not Call

A possible fix that works:

@@ -60,9 +60,9 @@ extra_pad_size = attrs.get('extra_pad_size', 0) if extra_pad_size: pad_width = ((0, 0), (0, 0), (0, extra_pad_size), (0, extra_pad_size)) - inputs = get_relay_op('pad')(*inputs, + inputs = [get_relay_op('pad')(*inputs, pad_width=pad_width, - pad_value=np.finfo(np.float32).min) + pad_value=np.finfo(np.float32).min)] return get_relay_op('max_pool2d')(*inputs, **new_attrs) def _darknet_avgpooling(inputs, params, attrs, prefix):

With this, as bonus, we just fixed issue #3057 partially using relay func instead of nnvm syms (yolov3-tiny, cuda version) !

python/tvm/relay/frontend/darknet.py

srkreddy1238 · 2019-05-13T05:32:55Z

tutorials/frontend/from_darknet.py

+# under the License.
+"""
+Compile YOLO-V2 and YOLO-V3 in DarkNet Models
+=================================


Doc formatting.

srkreddy1238 · 2019-05-13T05:33:13Z

tests/python/frontend/darknet/test_forward.py

+# under the License.
+"""
+Test Darknet Models
+=====================


Doc formatting.

srkreddy1238 · 2019-05-13T05:33:42Z

tutorials/frontend/from_darknet.py

+# Download cfg and weights file if first time.
+CFG_NAME = MODEL_NAME + '.cfg'
+WEIGHTS_NAME = MODEL_NAME + '.weights'
+REPO_URL = 'https://github.com/siju-samuel/darknet/blob/master/'


Use common location here too.

srkreddy1238 · 2019-05-13T05:34:03Z

tutorials/frontend/from_darknet.py

+[neth, netw] = shape['data'][2:] # Current image shape is 608x608
+######################################################################
+# Load a test image
+# --------------------------------------------------------------------


doc formatting

srkreddy1238 · 2019-05-13T05:34:19Z

tutorials/frontend/from_darknet.py

+# --------------------------------------------------------------------
+test_image = 'dog.jpg'
+print("Loading the test image...")
+img_url = 'https://github.com/siju-samuel/darknet/blob/master/data/' + \


Use from common location

srkreddy1238 · 2019-05-13T05:34:43Z

tutorials/frontend/from_darknet.py

+tvm.relay.testing.yolo_detection.do_nms_sort(dets, last_layer.classes, nms_thresh)
+
+coco_name = 'coco.names'
+coco_url = 'https://github.com/siju-samuel/darknet/blob/master/data/' + coco_name + '?raw=true'


common location here too.

tqchen · 2019-05-16T03:28:02Z

This PR has been relatively inactive. @siju-samuel please act based on review comments.

cbalint13 · 2019-05-22T14:49:29Z

@siju-samuel,

Can get this in ? After this PR i would add yolo-tiny variant with tuning to your example. I am getting 123ms on mali with float16, probably same on plain arm cpu too. Perhaps can get an android demo too after it.

siju-samuel · 2019-05-22T14:52:28Z

@cbalint13 i will update the PR by tomorrow. Tied up with some other work.

kazum · 2019-05-23T21:33:29Z

Sorry for being away for long time. The final version looks good to me.
@srkreddy1238 @jroesch @cbalint13, can you give approval if it looks good to you?

cbalint13 · 2019-05-23T21:41:31Z

Sorry for being away for long time. The final version looks good to me.
@srkreddy1238 @jroesch @cbalint13, can you give approval if it looks good to you?

I underlined a sigle functional bug in earlier stage of PR, was already fixed, have no more mentions. Glad to see the PR merged.

srkreddy1238

Thanks @siju-samuel. LGTM.

tqchen · 2019-05-24T17:18:38Z

will leave the PR management to @kazum

kazum · 2019-05-24T21:39:28Z

Thanks @siju-samuel @srkreddy1238 @cbalint13 @jroesch, this is now merged!

* [RELAY]Frontend darknet * CI test file updated & CI error fixed * avg_pool pad fix * Changed repo_url and doc formatting

apivovarov · 2021-07-14T21:46:34Z

@siju-samuel Yolov3 works fine, but yolov2 fails with OverflowError.
Related discussion: https://discuss.tvm.apache.org/t/yolov2-tutorial-overflowerror/10495

kazum requested changes Mar 12, 2019

View reviewed changes

siju-samuel force-pushed the relay_darknet branch from 1ebcdcd to d527784 Compare March 12, 2019 17:14

tqchen added the status: need review label Mar 14, 2019

jroesch requested a review from kazum March 17, 2019 02:55

jroesch reviewed Mar 17, 2019

View reviewed changes

kazum requested changes Mar 18, 2019

View reviewed changes

kazum requested changes Mar 30, 2019

View reviewed changes

kazum added status: need update need update based on feedbacks and removed status: need review labels Mar 30, 2019

kazum mentioned this pull request Mar 30, 2019

[DOCKER][FRONTEND] Run DarkNet tests #2673

Merged

yzhliu assigned kazum Apr 2, 2019

siju-samuel force-pushed the relay_darknet branch 7 times, most recently from 1987bec to fd0c6a3 Compare April 19, 2019 03:19

cbalint13 reviewed Apr 25, 2019

View reviewed changes

siju-samuel force-pushed the relay_darknet branch from 6172bde to 6748bc0 Compare May 2, 2019 03:03

srkreddy1238 requested changes May 13, 2019

View reviewed changes

siju-samuel mentioned this pull request May 22, 2019

Darknet lib added to webdata for use in dmlc/tvm dmlc/web-data#177

Merged

[RELAY]Frontend darknet

e567d89

siju-samuel added 3 commits May 23, 2019 08:45

CI test file updated & CI error fixed

4ad2d6d

avg_pool pad fix

c75ac97

Changed repo_url and doc formatting

13f7a75

siju-samuel force-pushed the relay_darknet branch from 6748bc0 to 13f7a75 Compare May 23, 2019 03:38

kazum approved these changes May 23, 2019

View reviewed changes

kazum added status: need review and removed status: need update need update based on feedbacks labels May 23, 2019

srkreddy1238 approved these changes May 24, 2019

View reviewed changes

kazum merged commit d3958e1 into apache:master May 24, 2019

yzhliu added status: accepted and removed status: need review labels May 25, 2019

wweic pushed a commit to wweic/tvm that referenced this pull request Jun 26, 2019

[RELAY]Frontend darknet (apache#2773)

1e66d2f

* [RELAY]Frontend darknet * CI test file updated & CI error fixed * avg_pool pad fix * Changed repo_url and doc formatting

wweic pushed a commit to neo-ai/tvm that referenced this pull request Jun 27, 2019

[RELAY]Frontend darknet (apache#2773)

7f70b8c

* [RELAY]Frontend darknet * CI test file updated & CI error fixed * avg_pool pad fix * Changed repo_url and doc formatting

tqchen mentioned this pull request Nov 8, 2019

[RELEASE][DRAFT] TVM v0.6 Release candidate #4259

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RELAY]Frontend darknet #2773

[RELAY]Frontend darknet #2773

siju-samuel commented Mar 11, 2019 •

edited

Loading

jroesch commented Mar 11, 2019

kazum Mar 12, 2019

jroesch commented Mar 17, 2019

jroesch Mar 17, 2019

kazum Mar 18, 2019

siju-samuel Apr 25, 2019

kazum Mar 18, 2019

kazum Mar 18, 2019

kazum Mar 18, 2019

kazum Mar 18, 2019

kazum Mar 18, 2019

yzhliu commented Mar 29, 2019

kazum Mar 30, 2019

siju-samuel commented Apr 25, 2019

cbalint13 Apr 25, 2019 •

edited

Loading

srkreddy1238 May 13, 2019

srkreddy1238 May 13, 2019

srkreddy1238 May 13, 2019

srkreddy1238 May 13, 2019

srkreddy1238 May 13, 2019

srkreddy1238 May 13, 2019

tqchen commented May 16, 2019

cbalint13 commented May 22, 2019

siju-samuel commented May 22, 2019

kazum commented May 23, 2019

cbalint13 commented May 23, 2019

srkreddy1238 left a comment

tqchen commented May 24, 2019

kazum commented May 24, 2019

apivovarov commented Jul 14, 2021

[RELAY]Frontend darknet #2773

[RELAY]Frontend darknet #2773

Conversation

siju-samuel commented Mar 11, 2019 • edited Loading

jroesch commented Mar 11, 2019

Choose a reason for hiding this comment

jroesch commented Mar 17, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yzhliu commented Mar 29, 2019

Choose a reason for hiding this comment

siju-samuel commented Apr 25, 2019

cbalint13 Apr 25, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tqchen commented May 16, 2019

cbalint13 commented May 22, 2019

siju-samuel commented May 22, 2019

kazum commented May 23, 2019

cbalint13 commented May 23, 2019

srkreddy1238 left a comment

Choose a reason for hiding this comment

tqchen commented May 24, 2019

kazum commented May 24, 2019

apivovarov commented Jul 14, 2021

siju-samuel commented Mar 11, 2019 •

edited

Loading

cbalint13 Apr 25, 2019 •

edited

Loading