Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BYOC][ACL] Support asymmetric per-layer quantized operators #6109

Merged
merged 4 commits into from
Jul 29, 2020
Merged

[BYOC][ACL] Support asymmetric per-layer quantized operators #6109

merged 4 commits into from
Jul 29, 2020

Conversation

lhutton1
Copy link
Contributor

@lhutton1 lhutton1 commented Jul 22, 2020 •
edited
Loading

Adds support for asymmetric per-layer quantization in the Arm Compute Library runtime module. This includes support for qnn.conv2d, nn.maxpool2d and reshape.

  • Adds support for different datatypes (as opposed to hard-coded fp32).
  • Adds support for creating ACL tensor with scale and offset values.
  • Improved qnn.conv2d layout conversion to support different kernel layouts.
  • Added qnn.conv2d composite operator (pad?, qnn.conv2d, bias?, relu?, qnn.requantize).
  • Updated tests to reflect these changes.
  • Added table of supported operators in tutorial.

cc @mbaret @manupa-arm @u99127 @Leo-arm @comaniac @zhiics

comaniac
comaniac requested changes Jul 22, 2020
View reviewed changes
Copy link
Contributor

@comaniac comaniac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also cc @anijain2305 to review layout_conversions in qnn.

src/relay/backend/contrib/arm_compute_lib/codegen.cc Outdated Show resolved Hide resolved
src/relay/backend/contrib/arm_compute_lib/codegen.cc Show resolved Hide resolved
src/runtime/contrib/arm_compute_lib/acl_runtime.cc Outdated Show resolved Hide resolved
src/runtime/contrib/arm_compute_lib/acl_runtime.cc Outdated Show resolved Hide resolved
src/runtime/contrib/arm_compute_lib/acl_utils.cc Show resolved Hide resolved
src/runtime/contrib/arm_compute_lib/acl_utils.cc Outdated Show resolved Hide resolved
src/runtime/contrib/arm_compute_lib/acl_utils.cc Show resolved Hide resolved
src/runtime/contrib/arm_compute_lib/acl_utils.h Outdated Show resolved Hide resolved
tests/python/contrib/test_arm_compute_lib/test_conv2d.py Outdated Show resolved Hide resolved
lhutton1 added 3 commits July 24, 2020 10:53
@lhutton1
[BYOC][ACL] Support asymmetric per-layer quantization
7bdd494 
Adds support for asymmetric per-layer quantization in the ACL runtime. This includes support for qnn.conv2d, nn.maxpool2d and reshape. Reflected these changes in codegen and runtime tests.

Change-Id: I8f610bd37af1e3740fd48c2d502bcc4727d9d712
@lhutton1
Address comments
63574f6 
Change-Id: I4f9e3e7dbf6053066927cf07c4c19ecc88572e9d
@lhutton1
Fix tutorial
05016ad 
Change-Id: I4371e9d97a120fb7776db40ffcde60f46927af4d
@lhutton1 lhutton1 force-pushed the acl-quantization-support branch from 96f17ca to 05016ad Compare July 24, 2020 10:49
comaniac
comaniac requested changes Jul 27, 2020
View reviewed changes
Copy link
Contributor

@comaniac comaniac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very close to merge IMO.

cc @zhiics PTAL.

tests/python/contrib/test_arm_compute_lib/infrastructure.py Outdated Show resolved Hide resolved
@lhutton1
Improve test infrastructure
d3c08f2 
* Doc-string for generate trials
* Output params on error

Change-Id: Ib2e2b1fcdf05cdc77f7f4fb4b46395f28c129957
@zhiics zhiics merged commit 39c7036 into apache:master Jul 29, 2020
@zhiics
Copy link
Member

zhiics commented Jul 29, 2020

Thanks @lhutton1 @comaniac

trevor-m pushed a commit to trevor-m/tvm that referenced this pull request Aug 26, 2020
@lhutton1
[BYOC][ACL] Support asymmetric per-layer quantized operators (apache#…
ca7c0a7 
…6109)

* [BYOC][ACL] Support asymmetric per-layer quantization

Adds support for asymmetric per-layer quantization in the ACL runtime. This includes support for qnn.conv2d, nn.maxpool2d and reshape. Reflected these changes in codegen and runtime tests.

Change-Id: I8f610bd37af1e3740fd48c2d502bcc4727d9d712

* Address comments

Change-Id: I4f9e3e7dbf6053066927cf07c4c19ecc88572e9d

* Fix tutorial

Change-Id: I4371e9d97a120fb7776db40ffcde60f46927af4d

* Improve test infrastructure

* Doc-string for generate trials
* Output params on error

Change-Id: Ib2e2b1fcdf05cdc77f7f4fb4b46395f28c129957
trevor-m pushed a commit to trevor-m/tvm that referenced this pull request Aug 26, 2020
@lhutton1
[BYOC][ACL] Support asymmetric per-layer quantized operators (apache#…
4533b78 
…6109)

* [BYOC][ACL] Support asymmetric per-layer quantization

Adds support for asymmetric per-layer quantization in the ACL runtime. This includes support for qnn.conv2d, nn.maxpool2d and reshape. Reflected these changes in codegen and runtime tests.

Change-Id: I8f610bd37af1e3740fd48c2d502bcc4727d9d712

* Address comments

Change-Id: I4f9e3e7dbf6053066927cf07c4c19ecc88572e9d

* Fix tutorial

Change-Id: I4371e9d97a120fb7776db40ffcde60f46927af4d

* Improve test infrastructure

* Doc-string for generate trials
* Output params on error

Change-Id: Ib2e2b1fcdf05cdc77f7f4fb4b46395f28c129957
trevor-m pushed a commit to trevor-m/tvm that referenced this pull request Aug 26, 2020
@lhutton1
[BYOC][ACL] Support asymmetric per-layer quantized operators (apache#…
65b4b08 
…6109)

* [BYOC][ACL] Support asymmetric per-layer quantization

Adds support for asymmetric per-layer quantization in the ACL runtime. This includes support for qnn.conv2d, nn.maxpool2d and reshape. Reflected these changes in codegen and runtime tests.

Change-Id: I8f610bd37af1e3740fd48c2d502bcc4727d9d712

* Address comments

Change-Id: I4f9e3e7dbf6053066927cf07c4c19ecc88572e9d

* Fix tutorial

Change-Id: I4371e9d97a120fb7776db40ffcde60f46927af4d

* Improve test infrastructure

* Doc-string for generate trials
* Output params on error

Change-Id: Ib2e2b1fcdf05cdc77f7f4fb4b46395f28c129957
trevor-m pushed a commit to trevor-m/tvm that referenced this pull request Sep 2, 2020
@lhutton1
[BYOC][ACL] Support asymmetric per-layer quantized operators (apache#…
e0ca076 
…6109)

* [BYOC][ACL] Support asymmetric per-layer quantization

Adds support for asymmetric per-layer quantization in the ACL runtime. This includes support for qnn.conv2d, nn.maxpool2d and reshape. Reflected these changes in codegen and runtime tests.

Change-Id: I8f610bd37af1e3740fd48c2d502bcc4727d9d712

* Address comments

Change-Id: I4f9e3e7dbf6053066927cf07c4c19ecc88572e9d

* Fix tutorial

Change-Id: I4371e9d97a120fb7776db40ffcde60f46927af4d

* Improve test infrastructure

* Doc-string for generate trials
* Output params on error

Change-Id: Ib2e2b1fcdf05cdc77f7f4fb4b46395f28c129957
trevor-m pushed a commit to neo-ai/tvm that referenced this pull request Sep 3, 2020
@lhutton1 @trevor-m
[BYOC][ACL] Support asymmetric per-layer quantized operators (apache#…
ce2d1bd 
…6109)

* [BYOC][ACL] Support asymmetric per-layer quantization

Adds support for asymmetric per-layer quantization in the ACL runtime. This includes support for qnn.conv2d, nn.maxpool2d and reshape. Reflected these changes in codegen and runtime tests.

Change-Id: I8f610bd37af1e3740fd48c2d502bcc4727d9d712

* Address comments

Change-Id: I4f9e3e7dbf6053066927cf07c4c19ecc88572e9d

* Fix tutorial

Change-Id: I4371e9d97a120fb7776db40ffcde60f46927af4d

* Improve test infrastructure

* Doc-string for generate trials
* Output params on error

Change-Id: Ib2e2b1fcdf05cdc77f7f4fb4b46395f28c129957
@lhutton1 lhutton1 deleted the acl-quantization-support branch September 23, 2020 10:28
@ZihengJiang ZihengJiang mentioned this pull request Sep 25, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@comaniac comaniac comaniac requested changes

@zhiics zhiics zhiics approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@lhutton1 @zhiics @comaniac