Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP][QNN] Quantized fully connected #3597

Closed
wants to merge 56 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
8d9e317
[Relay] [Quantization] WIP - Common files for the qauntization work.
Jul 8, 2019
5485b58
[Relay] [Quantization] WIP - Prototyping requantize op.
Jul 8, 2019
877d834
Requantize operator implementation.
anijain2305 Jul 10, 2019
705b796
Typo and lint fixes.
anijain2305 Jul 10, 2019
6cd1328
Lint fix.
anijain2305 Jul 10, 2019
ac4349b
Doc fix.
anijain2305 Jul 10, 2019
a9fef75
Uncommenting the lint script (fixing mistake).
anijain2305 Jul 10, 2019
d9eff68
Modifying the unit tests.
anijain2305 Jul 10, 2019
abc7c4e
Moving C++ files into src/relay/qnn
anijain2305 Jul 11, 2019
275ddd0
Moving python files to python/tvm/relay/qnn. Some minor fixes.
anijain2305 Jul 11, 2019
a0ad8ca
Moving the attrs.h inside the include directory.
anijain2305 Jul 11, 2019
ff8936c
Pushing files that I forgot earlier. Changing util location.
anijain2305 Jul 11, 2019
bdca4c6
[Relay] [Quantization] WIP - Common files for the qauntization work.
Jul 8, 2019
755f934
[Relay] [Quantization] WIP - Prototyping requantize op.
Jul 8, 2019
dba71f0
Requantize operator implementation.
anijain2305 Jul 10, 2019
6016b2a
Typo and lint fixes.
anijain2305 Jul 10, 2019
d54cea8
Lint fix.
anijain2305 Jul 10, 2019
ca954e0
Doc fix.
anijain2305 Jul 10, 2019
db24f1e
Uncommenting the lint script (fixing mistake).
anijain2305 Jul 10, 2019
523e16a
Modifying the unit tests.
anijain2305 Jul 10, 2019
18bff76
Moving C++ files into src/relay/qnn
anijain2305 Jul 11, 2019
32b69df
Moving python files to python/tvm/relay/qnn. Some minor fixes.
anijain2305 Jul 11, 2019
21168ae
Moving the attrs.h inside the include directory.
anijain2305 Jul 11, 2019
4a4beec
Pushing files that I forgot earlier. Changing util location.
anijain2305 Jul 11, 2019
120c050
Incorporating comments. API change. Lint fixes.
anijain2305 Jul 15, 2019
989bbea
Modifying the GetFixedPointMultiplierShift API as per comments.
anijain2305 Jul 15, 2019
8df0ddb
Forgot the dialect change.
anijain2305 Jul 15, 2019
8d0af86
Retriggering Jenkins.
anijain2305 Jul 15, 2019
ff1b9e3
Changing rewrite to qnn_lower.
anijain2305 Jul 15, 2019
362869f
Renaming Quantize to Qnn for clarity.
anijain2305 Jul 15, 2019
36f0ed9
Remove use_int_domain.
anijain2305 Jul 17, 2019
b45c629
Working quantized fully-connected with int8 and uint8
Jul 17, 2019
419dee0
Merge branch 'requantize' into qfullyconnected
Jul 17, 2019
4958495
Incorportaing review comments.
anijain2305 Jul 19, 2019
f858a83
Adding API doc for QNN dialect.
anijain2305 Jul 19, 2019
823cc94
Move the qnn_lower pass to transform namespace.
anijain2305 Jul 19, 2019
28a9587
Moving from expr to module. Adding namespace in C++.
anijain2305 Jul 19, 2019
76476dc
Working test case for int/uint with bias_add
Jul 19, 2019
732d6ce
Minor sentence rewrites. Added qnn namespace.
anijain2305 Jul 19, 2019
fadc573
Added the API doc.
anijain2305 Jul 19, 2019
956d3de
Chanding default out_dtype to int8. Adding a test with in/out_dtype a…
anijain2305 Jul 19, 2019
7a63597
merge from upstream/requantize
Jul 19, 2019
3ffdbf8
Merge branch 'requantize' into qfullyconnected
Jul 19, 2019
d700945
Style fixes. Better error messages.
anijain2305 Jul 19, 2019
21963dc
Removing extra code.
Jul 22, 2019
29c9e06
Merge branch 'requantize' into qfullyconnected
Jul 22, 2019
d0fdd1c
Adding documentation.
anijain2305 Jul 22, 2019
33cc075
More documentation fixes.
anijain2305 Jul 22, 2019
bb38855
Adding out dtype check for requantize.
anijain2305 Jul 22, 2019
7aac28d
Adding corner case for FP32 to fixed point conversion.
anijain2305 Jul 22, 2019
635b053
Adding extra line.
anijain2305 Jul 22, 2019
222e189
Documentation fix.
anijain2305 Jul 22, 2019
6c833d5
quantized fully connected working with requantize.
Jul 22, 2019
a115c96
Adding static inline.
anijain2305 Jul 23, 2019
572a8f3
Merge branch 'master' into requantize
Jul 24, 2019
dd213b6
Merge branch 'requantize' into qfullyconnected
Jul 24, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Typo and lint fixes.
anijain2305 committed Jul 15, 2019

Verified

This commit was signed with the committer’s verified signature.
MaxymVlasov Maksym Vlasov
commit 6016b2a573818a83f2b23155486c03fb91cabfd9
6 changes: 3 additions & 3 deletions include/tvm/relay/attrs/qnn.h
Original file line number Diff line number Diff line change
@@ -21,8 +21,8 @@
* \file tvm/relay/attrs/nn.h
* \brief Auxiliary attributes for nn operators.
*/
#ifndef TVM_RELAY_ATTRS_NN_QUANTIZE_H_
#define TVM_RELAY_ATTRS_NN_QUANTIZE_H_
#ifndef TVM_RELAY_ATTRS_QNN_H_
#define TVM_RELAY_ATTRS_QNN_H_

#include <tvm/attrs.h>
#include <string>
@@ -67,4 +67,4 @@ struct RequantizeAttrs : public tvm::AttrsNode<RequantizeAttrs> {

} // namespace relay
} // namespace tvm
#endif // TVM_RELAY_ATTRS_NN_QUANTIZE_H_
#endif // TVM_RELAY_ATTRS_QNN_H_
27 changes: 15 additions & 12 deletions include/tvm/relay/quantize_util.h
Original file line number Diff line number Diff line change
@@ -22,10 +22,11 @@
* \brief Utility methods needs for quantized ops that can be shared
*/

#ifndef TVM_QUANTIZE_UTIL_H
#define TVM_QUANTIZE_UTIL_H
#ifndef TVM_RELAY_QUANTIZE_UTIL_H_
#define TVM_RELAY_QUANTIZE_UTIL_H_

#include <tvm/expr.h>
#include<limits>
#include "./base.h"

namespace tvm {
@@ -68,14 +69,15 @@ inline bool is_quantized_type(const DataType& dtype) {
}

enum class QuantizeOpType : uint8_t {
Quantize_Requantize,
Quantize,
Dequantize,
Requantize
};

inline bool is_valid_quantized_op_input_type(const QuantizeOpType &op_type, const DataType &in_dtype) {
switch(op_type) {
case QuantizeOpType::Quantize_Requantize:
inline bool is_valid_quantized_op_input_type(const QuantizeOpType &op_type,
const DataType &in_dtype) {
switch (op_type) {
case QuantizeOpType::Quantize:
return is_Float32(in_dtype) || is_quantized_type(in_dtype);
case QuantizeOpType ::Dequantize:
return is_quantized_type(in_dtype);
@@ -86,9 +88,10 @@ inline bool is_valid_quantized_op_input_type(const QuantizeOpType &op_type, cons
}
}

inline bool is_valid_quantized_op_output_type(const QuantizeOpType &op_type, const DataType &in_dtype) {
switch(op_type) {
case QuantizeOpType::Quantize_Requantize:
inline bool is_valid_quantized_op_output_type(const QuantizeOpType &op_type,
const DataType &in_dtype) {
switch (op_type) {
case QuantizeOpType::Quantize:
return is_quantized_type(in_dtype);
case QuantizeOpType::Dequantize:
return is_Float32(in_dtype);
@@ -134,6 +137,6 @@ inline const int32_t get_qmax(const DataType& dtype) {
return -1;
}

} // namespace relay
} // namespace tvm
#endif //TVM_QUANTIZE_UTIL_H
} // namespace relay
} // namespace tvm
#endif // TVM_RELAY_QUANTIZE_UTIL_H_
2 changes: 1 addition & 1 deletion python/tvm/relay/op/qnn/__init__.py
Original file line number Diff line number Diff line change
@@ -17,4 +17,4 @@
# pylint: disable=wildcard-import
"""Neural network related operators."""
from __future__ import absolute_import as _abs
from .qnn import *
from .qnn import *
4 changes: 2 additions & 2 deletions python/tvm/relay/op/qnn/qnn.py
Original file line number Diff line number Diff line change
@@ -20,8 +20,8 @@
from . import _make

def requantize(input_data, input_zero_point, input_scale, output_zero_point,
output_scale, out_dtype="int32", use_int_compute=False,
rounding_mode="FE_UPWARD"):
output_scale, out_dtype="int32", use_int_compute=False,
rounding_mode="FE_UPWARD"):
r"""Requantized operator.
The requantize operator converts one quantized tensor to another quantized
1 change: 0 additions & 1 deletion python/tvm/relay/quantize/rewrite.py
Original file line number Diff line number Diff line change
@@ -19,7 +19,6 @@
from __future__ import absolute_import

from . import _quantize
from .. import expr as _expr

def rewrite(expr):
"""
4 changes: 2 additions & 2 deletions src/relay/pass/quantize_rewrite.cc
Original file line number Diff line number Diff line change
@@ -93,7 +93,7 @@ void GetFixedPointMultiplierShift(double double_multiplier,
*
*/
Expr RequantizeInt(const Expr& input_tensor,
const RequantizeAttrs*& param, const DataType& idtype,
const RequantizeAttrs* param, const DataType& idtype,
const Array<IndexExpr>& out_shape) {

double double_multiplier = param->input_scale/param->output_scale;
@@ -183,7 +183,7 @@ Expr RequantizeInt(const Expr& input_tensor,
* the input_tensor, round to nearest integer and then cast back to int32.
*/
Expr RequantizeFloat(const Expr& input_tensor,
const RequantizeAttrs*& param, const DataType& idtype,
const RequantizeAttrs* param, const DataType& idtype,
const Array<IndexExpr>& out_shape) {
double double_multiplier = param->input_scale/param->output_scale;
auto scalar_multiplier = MakeConstantScalar(Float(32), double_multiplier);
1 change: 0 additions & 1 deletion tests/python/unittest/test_quantized_ops.py
Original file line number Diff line number Diff line change
@@ -61,7 +61,6 @@ def get_func(data_shape, data_dtype, out_dtype, use_int_compute,
func)
func = run_infer_type(func)
func = relay.quantize.rewrite(func)
print(func)
return func