Summarize operators used in ConvS2S

[lcy-seso]

Here I summarize operators will be used in ConvS2S:

1. positional embedding

   • look_up_table but has to support padding_idx : Support padding_idx in the lookup_table_op. #7309
   • addition

2. convolution block structure: one-dimensional convolution followed by a GLU.

   Is it necessary to implement GLU in one operator to optimize the time efficiency. This can be determined later.

   • sequence convolution
     • 2D convolution: sequence_conv_op
   • GLU
     • offset operator ?? (To be determined later)
     • sigmoid
     • element-wise multiplication
     • addition
   • attention
     • matmul_op (the batched matrix multiplication)
     • softmax along the specified axis
     • reshape op
     • softmax
   • weight normalization

---

The missing operator:

1. weight normalization, related to Implement weight normalization in Fluid. #6914

Need to be enhanced: (This enhancement is also needed by both Transformer and ConvS2S)

1. look_up_table : related to Support padding_idx in the lookup_table_op. #7309