From 65c957e5c2d873c1003793a426e7bfeab464b57f Mon Sep 17 00:00:00 2001 From: Roman Kazantsev Date: Sat, 11 Jul 2020 00:50:01 +0500 Subject: [PATCH] Correct documentation for CTCLoss after #1 review Signed-off-by: Roman Kazantsev --- docs/ops/sequence/CTCLoss_4.md | 55 ++++++++++++++++++++-------------- 1 file changed, 33 insertions(+), 22 deletions(-) diff --git a/docs/ops/sequence/CTCLoss_4.md b/docs/ops/sequence/CTCLoss_4.md index 6b6effe760f7db..2288413322d998 100644 --- a/docs/ops/sequence/CTCLoss_4.md +++ b/docs/ops/sequence/CTCLoss_4.md @@ -10,14 +10,13 @@ This operation is similar to the TensorFlow* operation [CTCLoss](https://www.tensorflow.org/api_docs/python/tf/compat/v1/nn/ctc_loss) -*CTCLoss* calculates loss between input and target sequences. It sums over the probability of possible alignments of input to target. +*CTCLoss* estimates a chance that a target can occur (or is real) for given input sequence of logits. +Briefly, *CTCLoss* operation finds all sequences aligned with a target sequence `labels[i,:]`, computes log-probabilities of these aligned sequences using `inputs[:,i,:]` of logits +and computes a negative sum of these log-probabilies. -Input sequences `inputs[:,i,:]` can have different length. The lengths of sequences are coded as values 1 and 0 in the second input tensor `sequence_mask`. -Value `sequence_mask[j, i]` specifies whether there is a sequence symbol at index `j` in the sequence `i` in the batch of sequences. -If there is no symbol at `j`-th position `sequence_mask[j, i] = 0`, and `sequence_mask[j, i] = 1` otherwise. -Starting from `j = 0`, `sequence_mass[j, i]` are equal to 1 up to the particular index `j = last_sequence_symbol`, -which is defined independently for each sequence `i`. For `j > last_sequence_symbol`, values in `sequence_mask[j, i]` are all zeros. -A length of target sequence from `labels[i,:]` must not be greater than a lenght of corresponding input sequence. Otherwise, the operation behaviour is undefined. +Input sequences of logits from `inputs` can have different length. The length of each sequence `inputs[:,i,:]` equals `input_length[i]`. +A length of target sequence from `labels[i,:]` is determined by a pad `-1`. The length must not be greater than a lenght of corresponding input sequence `inputs[:,i,:]`. +Otherwise, the operation behaviour is undefined. *CTCLoss* calculation scheme: @@ -29,18 +28,19 @@ p_{t,i,j} = \frac{\exp(inputs[t,i,j])}{\sum^{K}_{k=0}{\exp(inputs[t,i,k])}} 2. For a given `i`-th target from `labels[i,:]` find all aligned paths. A path `S = (c1,c2,...,cT)` is aligned with a target `G=(g1,g2,...,gT)` if both chains are equal after decoding. The decoding removes pads `-1` from a target `G` and merges repeated characters in `G` in case *preprocess_collapse_repeated* equal to True. -The decoding merges repeated characters in `S` in case *merge_repeated* equal to True and removes blank characters represented by last class index `K-1`, -where `K` is a number of classes including a blank symbol. -For example, in case default *merge_repeated* and *preprocess_collapse_repeated* a target sequence `(0,3,2,2,-1,-1,-1,-1,-1)` is processed to `(0,3,2,2)` and -an input `(0,0,4,3,2,2,4,2,4)` is also processed to `(0,3,2,2)`, where `K=5`. +The decoding merges repeated characters in `S` in case *ctc_merge_repeated* equal to True and removes blank characters represented by `blank_index`. +By default, `blank_index` is equal to `C-1`, where `C` is a number of classes including the blank. +For example, in case default *ctc_merge_repeated*, *preprocess_collapse_repeated*, *unique*, and `blank_index` a target sequence `(0,3,2,2,-1,-1,-1,-1,-1)` is processed to `(0,3,2,2)` and +a path `(0,0,4,3,2,2,4,2,4)` is also processed to `(0,3,2,2)`, where `C=5`. There exist other paths that are also aligned, for instance, `0,4,3,3,2,4,2,2,2`. +Paths checked for alignment with `label[:,i]` must be of length `input_length[i] = L_i`. Compute probabilities of these alignments as follows: \f[ -p(S) = \prod_{t=1}^{T} p_{t,i,ct} +p(S) = \prod_{t=1}^{L_i} p_{t,i,ct} \f] -3. Finally sum up probabilities of all aligned paths for a given target and compute negative logarithm of it: +3. Finally sum up logarithms of probabilities of all aligned paths with negative sign: \f[ -CTCLoss = -\log \sum_{S} p(S) +CTCLoss = - \sum_{S} \ln p(S) \f] @@ -54,22 +54,32 @@ CTCLoss = -\log \sum_{S} p(S) * **Default value**: False * **Required**: *no* -* *merge_repeated* +* *ctc_merge_repeated* - * **Description**: *merge_repeated* is a flag for merging repeated labels during the CTC loss calculation. + * **Description**: *ctc_merge_repeated* is a flag for merging repeated labels during the CTC loss calculation. * **Range of values**: True or False * **Type**: `boolean` * **Default value**: True * **Required**: *no* +* *unique* + + * **Description**: *unique* is a flag to find unique elements for each `labels[i,:]` before matching with potential alignments2. + * **Range of values**: True or False + * **Type**: `boolean` + * **Default value**: False + * **Required**: *no* + **Inputs** -* **1**: `inputs` - Input tensor with a batch of sequences. Type of elements is *T_F*. Shape of the tensor is `[T, N, C]`, where `T` is the maximum sequence length, `N` is the batch size and `C` is the number of classes. Required. +* **1**: `inputs` - Input tensor with a batch of sequences. Type of elements is *T_F*. Shape of the tensor is `[T, N, C]`, where `T` is the maximum sequence length, `N` is the batch size and `C` is the number of classes including the blank. Required. -* **2**: `sequence_mask` - 2D input tensor of type *T_IND* with sequence masks for each sequence in the batch. Populated with values 0 and 1. Shape of this input is `[T, N]`. Required. +* **2**: `input_length` - 1D input tensor of type *T_IND* and of a shape `[N]`. The tensor must consist of values not greater than `T`. Lengths of input sequences `inputs[:,i,:]`. Required. * **3**: `labels` - 2D tensor with shape `[N, T]` of type *T_IND*. A sequence can be shorter than the size `T` of the tensor, all elements that do not code sequence classes are filled with -1. Required. +* **4**: `blank_index` - Scalar. Set the class index to use for the blank label. Default value is `C-1`. Optional. + **Output** * **1**: Output tensor with shape `[N]`, negative log of summed up probabilities for aligned paths. Type of elements is *T_F*. @@ -78,7 +88,7 @@ CTCLoss = -\log \sum_{S} p(S) * *T_F*: any supported floating point type. -* *T_IND*: any supported integer type. +* *T_IND*: any supported signed integer type. **Example** @@ -89,7 +99,7 @@ CTCLoss = -\log \sum_{S} p(S) 20 8 128 - + 20 8 @@ -97,12 +107,13 @@ CTCLoss = -\log \sum_{S} p(S) 8 20 - + + 8 - + ```