add iou similarity operator #7566

wanghaox · 2018-01-16T10:44:25Z

resolve #7565

… iou_sim

qingqing01 · 2018-01-19T11:08:13Z

paddle/operators/iou_similarity_op.cc

+    auto x_dims = ctx->GetInputDim("X");
+    auto y_dims = ctx->GetInputDim("Y");
+
+    PADDLE_ENFORCE_EQ(x_dims.size(), 2UL, "The shape of X is [N, 4]");


The rank of Input(X) must be 2.

qingqing01 · 2018-01-19T11:09:36Z

paddle/operators/iou_similarity_op.cc

+  using framework::OperatorWithKernel::OperatorWithKernel;
+
+ protected:
+  void InferShape(framework::InferShapeContext *ctx) const override {


PADDLE_ENFORCE(ctx->HasInput("X"), "Input(X) of IOUSimilarityOp should not be null."); PADDLE_ENFORCE(ctx->HasInput("Y"), "Input(Y) of IOUSimilarityOp should not be null.");

qingqing01 · 2018-01-19T11:09:59Z

paddle/operators/iou_similarity_op.cc

+
+    PADDLE_ENFORCE_EQ(x_dims.size(), 2UL, "The shape of X is [N, 4]");
+    PADDLE_ENFORCE_EQ(x_dims[1], 4UL, "The shape of X is [N, 4]");
+    PADDLE_ENFORCE_EQ(y_dims.size(), 2UL, "The shape of Y is [M, 4]");


The rank of Input(Y) must be 2.

qingqing01 · 2018-01-19T11:11:57Z

paddle/operators/iou_similarity_op.cc

+    AddInput(
+        "X",
+        "(Tensor, default Tensor<float>) "
+        "BoxList X holding N boxes, each box is "


Box list X holds ...

qingqing01 · 2018-01-19T11:12:25Z

paddle/operators/iou_similarity_op.cc

+        "X",
+        "(Tensor, default Tensor<float>) "
+        "BoxList X holding N boxes, each box is "
+        "represented as [xmin, ymin, xmax, ymax], the shape of X is [N, 4].");


Better to explain the meaning of xmin, ymin, xmax, ymax

qingqing01 · 2018-01-19T11:13:21Z

paddle/operators/iou_similarity_op.cc

+
+    AddComment(R"DOC(
+IOU Similarity Operator.
+Computes pairwise intersection-over-union between box collections.


intersection-over-union (IOU) between two box lists.

qingqing01 · 2018-01-19T11:16:06Z

paddle/operators/iou_similarity_op.h

+
+    platform::ForRange<DeviceContext> for_range(
+        static_cast<const DeviceContext&>(ctx.device_context()), x_n);
+    for_range(functor);


platform::ForRange support GPU, please register GPU kernel.

qingqing01 · 2018-01-19T11:17:09Z

python/paddle/v2/fluid/tests/test_iou_similarity_op.py

+        self.check_output()
+
+    def test_check_grad(self):
+        return


Remove test_check_grad

qingqing01 · 2018-01-19T11:17:55Z

python/paddle/v2/fluid/tests/test_iou_similarity_op.py

+
+    def setUp(self):
+        self.op_type = "iou_similarity"
+        self.set_data()


If only one test, the code in set_data() and init_test_data() can be moved here.

qingqing01 · 2018-01-19T11:19:07Z

python/paddle/v2/fluid/tests/test_iou_similarity_op.py

+                                [0.0, 0.0, 20.0, 20.0]]).astype('float32')
+        self.output = np.array(
+            [[2.0 / 16.0, 0, 6.0 / 400.0],
+             [1.0 / 16.0, 0.0, 5.0 / 400.0]]).astype('float32')


Better to use random data and calculation the IOU in Python.

It's better to calculate in Python, but this version uses data to verify the run first.

… iou_sim

pkuyym · 2018-01-23T03:35:40Z

paddle/operators/iou_similarity_op.cu

+See the License for the specific language governing permissions and
+limitations under the License. */
+
+#define EIGEN_USE_GPU


Is this necessary?

pkuyym · 2018-01-23T03:46:30Z

paddle/operators/iou_similarity_op.cc

+    PADDLE_ENFORCE_EQ(y_dims.size(), 2UL, "The rank of Input(Y) must be 2.");
+    PADDLE_ENFORCE_EQ(y_dims[1], 4UL, "The shape of Y is [M, 4]");
+
+    ctx->SetOutputDim("Out", framework::make_ddim({x_dims[0], y_dims[0]}));


Please consider 'X' as a LoDTensor. Here, LoD of 'out' should inherit from 'X'.

Add ctx->ShareLoD("X", /*->*/ "Out"); in the InferShape like : https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/mul_op.cc#L68

pkuyym · 2018-01-23T03:46:51Z

paddle/operators/iou_similarity_op.cc

+  IOUSimilarityOpMaker(OpProto *proto, OpAttrChecker *op_checker)
+      : OpProtoAndCheckerMaker(proto, op_checker) {
+    AddInput("X",
+             "(Tensor, default Tensor<float>) "


X should be a LoDTensor.

pkuyym · 2018-01-23T03:48:45Z

paddle/operators/iou_similarity_op.cc

+
+    AddComment(R"DOC(
+IOU Similarity Operator.
+Computes intersection-over-union (IOU) between two box lists.


The document is too simple. Please explain the function more clearly. 'X' should be a LoDTensor and 'Y' is a common Tensor, boxes in 'Y' are shared by all input images.

done, added the formula

pkuyym · 2018-01-23T03:50:58Z

paddle/operators/iou_similarity_op.h

+  T inter_xmax = xmax1 > xmax2 ? xmax2 : xmax1;
+  T inter_ymax = ymax1 > ymax2 ? ymax2 : ymax1;
+  T inter_xmin = xmin1 > xmin2 ? xmin1 : xmin2;
+  T inter_ymin = ymin1 > ymin2 ? ymin1 : ymin2;


Please use 'min' and 'max' to make the code more readable.

Std:: min can't run under GPU.

qingqing01 · 2018-01-23T07:13:35Z

paddle/operators/iou_similarity_op.cc

+    PADDLE_ENFORCE_EQ(y_dims.size(), 2UL, "The rank of Input(Y) must be 2.");
+    PADDLE_ENFORCE_EQ(y_dims[1], 4UL, "The shape of Y is [M, 4]");
+
+    ctx->SetOutputDim("Out", framework::make_ddim({x_dims[0], y_dims[0]}));


Add ctx->ShareLoD("X", /*->*/ "Out"); in the InferShape like : https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/mul_op.cc#L68

qingqing01 · 2018-01-23T07:14:24Z

paddle/operators/iou_similarity_op.cc

+             "[xmax, ymax] is the right upper coordinate of the box.");
+
+    AddOutput("Out",
+              "(LoDTensor or Tensor, the lod is same as input X) The output of "


LoDTensor or Tensor -> LoDTensor

qingqing01 · 2018-01-23T07:37:30Z

paddle/operators/iou_similarity_op.cc

+             "(LoDTensor, default LoDTensor<float>) "
+             "Box list X is a 2-D LoDTensor with shape [N, 4] holds N boxes, "
+             "each box is represented as [xmin, ymin, xmax, ymax], "
+             "the shape of X is [N, 4]. [xmin, ymin] is the lower left "


[xmin, ymin] is the left top coordinate of the box if the input is image feature map. They are close to the origin of the coordinate system.

Modify other places too.

qingqing01 · 2018-01-23T08:07:50Z

paddle/operators/iou_similarity_op.cc

+IOU Similarity Operator.
+Computes intersection-over-union (IOU) between two box lists.
+ Box list 'X' should be a LoDTensor and 'Y' is a common Tensor,
+ boxes in 'Y' are shared by all input images.


by all instance of the batched inputs of X.

qingqing01 · 2018-01-23T08:12:55Z

paddle/operators/iou_similarity_op.cc

+Computes intersection-over-union (IOU) between two box lists.
+ Box list 'X' should be a LoDTensor and 'Y' is a common Tensor,
+ boxes in 'Y' are shared by all input images.
+ Given two box A and B, the calculation of IOU is as follows:


Given two boxes of A and B,

… iou_sim

qingqing01 · 2018-01-23T10:40:54Z

paddle/operators/iou_similarity_op.cc

+    AddInput("Y",
+             "(Tensor, default Tensor<float>) "
+             "Box list Y holds M boxes, each box is represented as "
+             "[xmin, ymin, xmax, ymax], the shape of X is [N, 4]. "


the shape of X is [N, 4] -> the shape of X is [M, 4]

qingqing01 · 2018-01-23T10:48:46Z

paddle/operators/iou_similarity_op.h

+    T y_min1 = x_[row_id * 4 + 1];
+    T x_max1 = x_[row_id * 4 + 2];
+    T y_max1 = x_[row_id * 4 + 3];
+    for (size_t i = 0; i < cols_; ++i) {


Here, cols_ is the number of prior_box, in the SSD this number is about 8732 or more, so, this is less efficient on GPU. This will be fixed later.

wanghaox requested review from pkuyym and qingqing01 January 16, 2018 10:46

wanghaox added 2 commits January 16, 2018 20:35

add iou similarity operator

2ad5a6f

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

3b63815

… iou_sim

qingqing01 mentioned this pull request Jan 17, 2018

The TODO lists for MobileNet-SSD model. #7488

Closed

25 tasks

qingqing01 reviewed Jan 19, 2018

View reviewed changes

wanghaox added 4 commits January 22, 2018 16:26

update iou_sim code

528bcac

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

3c255ad

… iou_sim

update code

0b178b8

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

d458795

… iou_sim

pkuyym reviewed Jan 23, 2018

View reviewed changes

pkuyym requested changes Jan 23, 2018

View reviewed changes

update code and doc, change input x to LoDTensor

a05d25c

qingqing01 reviewed Jan 23, 2018

View reviewed changes

wanghaox added 2 commits January 23, 2018 16:53

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

25a0193

… iou_sim

update code

fa10f03

qingqing01 approved these changes Jan 23, 2018

View reviewed changes

pkuyym approved these changes Jan 23, 2018

View reviewed changes

wanghaox merged commit d43932c into PaddlePaddle:develop Jan 23, 2018

wanghaox deleted the iou_sim branch January 23, 2018 11:24

qingqing01 mentioned this pull request Jan 24, 2018

Optimize the CUDA implementation of IOU operator. #7808

Closed

add iou similarity operator #7566

add iou similarity operator #7566

Conversation

wanghaox commented Jan 16, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qingqing01 Jan 23, 2018 • edited Loading

Choose a reason for hiding this comment

qingqing01 Jan 23, 2018 •

edited

Loading