[Frontend][TFlite] Add parser support for relu6, leaky_relu, relu_n1_to_1, log_softmax #4805

inadob · 2020-02-03T09:58:54Z

This patch is made on top of #4789.

Add helper functions in the parser to quantize and dequantize with Relay
Add qnn tests for all 4 activation functions

anijain2305

Relu and Clip implementation does not look right.

We can keep the computation in integer domain. The way to do that is to subtract the input zero point, and then call Relu, then requantize to the output scale (only if output scale/zero point are different from input scale/zero point).

wyc-ruiker · 2020-02-04T15:03:02Z

python/tvm/relay/frontend/tflite.py

+        return out
+
+    def convert_leaky_relu(self, op):
+        """Convert TFLite LEAKY_RELU"""


Not important, I think it should be """Convert TFLite Leaky_ReLU""" to align with """One iteration of Leaky_ReLU""".

wyc-ruiker · 2020-02-04T15:04:28Z

python/tvm/relay/frontend/tflite.py

+        return out
+
+    def convert_relu_n1_to_1(self, op):
+        """Convert TFLite RELU_N1_TO_1"""


Not important, I think it should be """Convert TFLite ReLU_n1_to_1""" to align with """One iteration of ReLU_n1_to_1""".

tqchen · 2020-02-26T23:42:09Z

@inadob please followup to address the review comments :)

inadob · 2020-02-28T11:56:49Z

Relu and Clip implementation does not look right.

We can keep the computation in integer domain. The way to do that is to subtract the input zero point, and then call Relu, then requantize to the output scale (only if output scale/zero point are different from input scale/zero point).

That's fine for the standard ReLU op but I am not entirely sure whether we can follow this logic when we apply ReLU6 and ReLU1. Is there a way to recreate them in Relay without clip?.

What I did, in this case, was to shift the data by subtracting the zero point, do clip, shift back and finally requantize if needed. The problem here was that Clip() from RELAY_PASS_PATTERN needs the clipping range to be in double and I couldn't just fix this by casting to float64 (it seems double is not supported in TVM).

@anijain @u99127

anijain2305 · 2020-02-28T19:18:39Z

The problem here was that Clip() from RELAY_PASS_PATTERN needs the clipping range to be in double and I couldn't just fix this by casting to float64 (it seems double is not supported in TVM).

Can you show me code example here? I have typically used float numbers like 1.0 or 6.0 to work with a_min or a_max

inadob · 2020-03-05T17:14:57Z

Can you show me code example here? I have typically used float numbers like 1.0 or 6.0 to work with a_min or a_max

Expr ClipQnnCanonicalize(const Attrs& attrs, const Array<Expr>& new_args,
                               const Array<tvm::relay::Type>& arg_types) {
  CHECK_EQ(new_args.size(), 7);
  auto& input_tensor = new_args[0];
  auto& input_scale = new_args[1]; // in fp
  auto& input_zero_point = new_args[2]; // in int32
  auto& clip_min = new_args[3];  // value is in fp
  auto& clip_max = new_args[4];  // value is in fp
  auto& output_scale = new_args[5];
  auto& output_zero_point = new_args[6];

  // Get the input dtype and shape.
  CHECK_EQ(arg_types.size(), 8);
  auto tensor_type = arg_types[0].as<TensorTypeNode>();
  CHECK(tensor_type != nullptr);
  auto input_dtype = tensor_type->dtype;
  auto input_shape = tensor_type->shape;


  // shift the input by subtracting the input zero_point
  auto shifted_input = Subtract(Cast(input_tensor, DataType::Int(32)), input_zero_point);

  // do the clipping in int32
  // auto clipped_tensor = Clip(shifted_input, clip_min, clip_max)
  auto clipped_tensor = Clip(shifted_input, Cast(clip_min, DataType::Float(64)), Cast(clip_max, DataType::Float(64)))
  // shift the input back by adding the zero_point
  clipped_tensor = Add(clipped_tensor, input_zero_point);

  // requantize the output if needed
  auto requantized_output = clipped_tensor;
  if (!IsEqualScalar(input_scale, output_scale) ||
      !IsEqualScalar(input_zero_point, output_zero_point)) {
     requantized_output = Requantize(clipped_tensor, input_shape, input_scale, input_zero_point, output_scale,
                                     output_zero_point, DataType::Int(32));
    }

  // Go back to lower precision.
  auto q_min = GetQmin(input_dtype);
  auto q_max = GetQmax(input_dtype);
  requantized_output = Clip(requantized_output, q_min, q_max); 
  return Cast(requantized_output, input_dtype);
}

And I am getting a complaint about double dtype...

/workspace/src/relay/qnn/op/clip.cc:61:117: error: cannot convert 'tvm::relay::Expr {aka tvm::RelayExpr}' to 'double' for argument '2' to 'tvm::relay::Expr tvm::relay::Clip(tvm::relay::Expr, double, double)'
   auto clipped_tensor = Clip(shifted_input, Cast(clip_min, DataType::Float(64)), Cast(clip_max, DataType::Float(64)))

The commented out line where I directly do clip() without casting to float64 didn't work too.

tqchen · 2020-03-30T20:52:16Z

@anijain2305 @inadob please followup :)

tqchen · 2020-04-24T02:55:55Z

ping

inadob · 2020-04-28T17:05:40Z

@anijain can you please help me with the quantized operators

anijain2305 · 2020-05-05T16:45:34Z

Sorry for the delay. @inadob Can you take a look at this PR - #5479

I have added support for fused quantized activations there. We dont need to create a new op in C++ from scratch. It is pretty easy to do the same thing in python. Let me know what you think

python/tvm/relay/frontend/tflite.py

anijain2305 · 2020-05-30T01:04:30Z

Ping @inadob Let us know if you are working on this. Or else I can take a chance at this.

inadob · 2020-06-01T00:03:42Z

Ping @inadob Let us know if you are working on this. Or else I can take a chance at this.

@anijain2305 Yes, I am working on it - I will be ready in the next few days. Sorry for the delay

anijain2305 · 2020-06-04T17:05:19Z

@inadob Your changes look better now. Can you please rebase? (git submodule update --init --recursive)

anijain2305 · 2020-06-04T19:33:37Z

It seems you still have old `3rdparty/dmlc-core'. You can check that by clicking on "Files changed" tab.

inadob · 2020-06-05T14:06:04Z

It seems you still have old `3rdparty/dmlc-core'. You can check that by clicking on "Files changed" tab.

It's fixed now.

anijain2305 · 2020-06-05T21:38:41Z

@inadob Can you please fix the CI error?

anijain2305 · 2020-06-10T00:34:13Z

Ping @inadob :)

tqchen · 2020-06-12T15:27:57Z

Glad that it is moving, @anijain2305 please manage this PR

…g_softmax * add implementation in parser * add qnn tests for each operator

* add 'clip' as in the quantized fused operations * remove redundant assertions and imports

inadob · 2020-06-16T16:47:27Z

@anijain2305
CI has been fixed :)

anijain2305 · 2020-06-16T17:16:48Z

Thanks @inadob This is merged!

…to_1, log_softmax (apache#4805) * [Frontend][TFLite]Add support for relu6, leaky_relu, relu_n1_to_1, log_softmax * add implementation in parser * add qnn tests for each operator * Implement clip operation for quantized relu6, relu1 * add 'clip' as in the quantized fused operations * remove redundant assertions and imports * Fix floating value quantization for RELU6 and RELU1

inadob requested a review from FrozenGene February 3, 2020 09:59

inadob changed the title ~~[Frontend][TFlite] Add parser support for relu6, leeaky_relu, relu_n1_to_1, log_softmax~~ [Frontend][TFlite] Add parser support for relu6, leaky_relu, relu_n1_to_1, log_softmax Feb 3, 2020

inadob mentioned this pull request Feb 3, 2020

[Relay][Frontend][TFlite] Add support for quantized LOGISTIC #4696

Merged

anijain2305 requested changes Feb 3, 2020

View reviewed changes

wyc-ruiker reviewed Feb 4, 2020

View reviewed changes

tqchen added the status: need review label Feb 6, 2020

tqchen assigned FrozenGene Feb 6, 2020

tqchen added the status: need update need update based on feedbacks label Feb 12, 2020

inadob mentioned this pull request Mar 2, 2020

[TFLITE]Activation functions support #4978

Merged

anijain2305 reviewed May 5, 2020

View reviewed changes

python/tvm/relay/frontend/tflite.py Outdated Show resolved Hide resolved

inadob force-pushed the activation branch from e02ad30 to a7b1c21 Compare June 3, 2020 17:30

inadob closed this Jun 3, 2020

inadob reopened this Jun 3, 2020

inadob force-pushed the activation branch from a7b1c21 to df59ded Compare June 4, 2020 12:16

inadob requested a review from anijain2305 June 4, 2020 14:51

inadob force-pushed the activation branch from df59ded to 91c3631 Compare June 4, 2020 18:11

inadob force-pushed the activation branch from 91c3631 to 39a83c3 Compare June 5, 2020 14:03

tqchen removed the status: need review label Jun 12, 2020

tqchen assigned anijain2305 Jun 12, 2020

inadob added 2 commits June 13, 2020 13:25

[Frontend][TFLite]Add support for relu6, leaky_relu, relu_n1_to_1, lo…

38c7445

…g_softmax * add implementation in parser * add qnn tests for each operator

Implement clip operation for quantized relu6, relu1

7f0599c

* add 'clip' as in the quantized fused operations * remove redundant assertions and imports

inadob force-pushed the activation branch 2 times, most recently from 4781b9e to 126361c Compare June 13, 2020 16:45

Fix floating value quantization for RELU6 and RELU1

9345405

inadob force-pushed the activation branch from 126361c to 9345405 Compare June 16, 2020 12:19

anijain2305 approved these changes Jun 16, 2020

View reviewed changes

anijain2305 merged commit 52bf113 into apache:master Jun 16, 2020

tqchen added status: accepted and removed status: need update need update based on feedbacks labels Jun 16, 2020

ZihengJiang mentioned this pull request Sep 25, 2020

TVM v0.7 Release Note Candidate #6486

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Frontend][TFlite] Add parser support for relu6, leaky_relu, relu_n1_to_1, log_softmax #4805

[Frontend][TFlite] Add parser support for relu6, leaky_relu, relu_n1_to_1, log_softmax #4805

inadob commented Feb 3, 2020

anijain2305 left a comment •

edited

Loading

wyc-ruiker Feb 4, 2020

wyc-ruiker Feb 4, 2020

tqchen commented Feb 26, 2020

inadob commented Feb 28, 2020 •

edited

Loading

anijain2305 commented Feb 28, 2020

inadob commented Mar 5, 2020

tqchen commented Mar 30, 2020

tqchen commented Apr 24, 2020

inadob commented Apr 28, 2020

anijain2305 commented May 5, 2020 •

edited

Loading

anijain2305 commented May 30, 2020

inadob commented Jun 1, 2020 •

edited

Loading

anijain2305 commented Jun 4, 2020

anijain2305 commented Jun 4, 2020

inadob commented Jun 5, 2020

anijain2305 commented Jun 5, 2020

anijain2305 commented Jun 10, 2020

tqchen commented Jun 12, 2020

inadob commented Jun 16, 2020

anijain2305 commented Jun 16, 2020

[Frontend][TFlite] Add parser support for relu6, leaky_relu, relu_n1_to_1, log_softmax #4805

[Frontend][TFlite] Add parser support for relu6, leaky_relu, relu_n1_to_1, log_softmax #4805

Conversation

inadob commented Feb 3, 2020

anijain2305 left a comment • edited Loading

Choose a reason for hiding this comment

wyc-ruiker Feb 4, 2020

Choose a reason for hiding this comment

wyc-ruiker Feb 4, 2020

Choose a reason for hiding this comment

tqchen commented Feb 26, 2020

inadob commented Feb 28, 2020 • edited Loading

anijain2305 commented Feb 28, 2020

inadob commented Mar 5, 2020

tqchen commented Mar 30, 2020

tqchen commented Apr 24, 2020

inadob commented Apr 28, 2020

anijain2305 commented May 5, 2020 • edited Loading

anijain2305 commented May 30, 2020

inadob commented Jun 1, 2020 • edited Loading

anijain2305 commented Jun 4, 2020

anijain2305 commented Jun 4, 2020

inadob commented Jun 5, 2020

anijain2305 commented Jun 5, 2020

anijain2305 commented Jun 10, 2020

tqchen commented Jun 12, 2020

inadob commented Jun 16, 2020

anijain2305 commented Jun 16, 2020

anijain2305 left a comment •

edited

Loading

inadob commented Feb 28, 2020 •

edited

Loading

anijain2305 commented May 5, 2020 •

edited

Loading

inadob commented Jun 1, 2020 •

edited

Loading