-
Notifications
You must be signed in to change notification settings - Fork 2.4k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fixing formatting and reference issues
- Loading branch information
1 parent
21bf6ab
commit 2b3ed45
Showing
44 changed files
with
521 additions
and
454 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
135 changes: 0 additions & 135 deletions
135
...nvino-ir-format/operation-sets/operations-specifications/internal/augru-cell.md
This file was deleted.
Oops, something went wrong.
157 changes: 157 additions & 0 deletions
157
...vino-ir-format/operation-sets/operations-specifications/internal/augru-cell.rst
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,157 @@ | ||
.. {#openvino_docs_ops_internal_AUGRUCell} | ||
AUGRUCell | ||
========= | ||
|
||
**Versioned name**: *AUAUGRUCell* | ||
|
||
**Category**: *Sequence processing* | ||
|
||
**Short description**: *AUGRUCell* represents a single AUGRU Cell (GRU with attentional | ||
update gate). | ||
|
||
**Detailed description**: The main difference between *AUGRUCell* and | ||
:doc:`GRUCell <../sequence/gru-cell-3>` is the additional attention score | ||
input ``A``, which is a multiplier for the update gate. | ||
The AUGRU formula is based on the `paper arXiv:1809.03672 <https://arxiv.org/abs/1809.03672>`__. | ||
|
||
.. code-block:: py | ||
AUGRU formula: | ||
* - matrix multiplication | ||
(.) - Hadamard product (element-wise) | ||
f, g - activation functions | ||
z - update gate, r - reset gate, h - hidden gate | ||
a - attention score | ||
rt = f(Xt*(Wr^T) + Ht-1*(Rr^T) + Wbr + Rbr) | ||
zt = f(Xt*(Wz^T) + Ht-1*(Rz^T) + Wbz + Rbz) | ||
ht = g(Xt*(Wh^T) + (rt (.) Ht-1)*(Rh^T) + Rbh + Wbh) # 'linear_before_reset' is False | ||
zt' = (1 - at) (.) zt # multiplication by attention score | ||
Ht = (1 - zt') (.) ht + zt' (.) Ht-1 | ||
**Attributes** | ||
* *hidden_size* | ||
* **Description**: *hidden_size* specifies hidden state size. | ||
* **Range of values**: a positive integer | ||
* **Type**: ``int`` | ||
* **Required**: *yes* | ||
* *activations* | ||
* **Description**: activation functions for gates | ||
* **Range of values**: *sigmoid*, *tanh* | ||
* **Type**: a list of strings | ||
* **Default value**: *sigmoid* for f, *tanh* for g | ||
* **Required**: *no* | ||
* *activations_alpha, activations_beta* | ||
* **Description**: *activations_alpha, activations_beta* attributes of functions; | ||
applicability and meaning of these attributes depends on chosen activation functions | ||
* **Range of values**: [] | ||
* **Type**: ``float[]`` | ||
* **Default value**: [] | ||
* **Required**: *no* | ||
* *clip* | ||
* **Description**: *clip* specifies bound values *[-C, C]* for tensor clipping. | ||
Clipping is performed before activations. | ||
* **Range of values**: ``0.`` | ||
* **Type**: ``float`` | ||
* **Default value**: ``0.`` that means the clipping is not applied | ||
* **Required**: *no* | ||
* *linear_before_reset* | ||
* **Description**: *linear_before_reset* flag denotes, if the output of hidden gate | ||
is multiplied by the reset gate before or after linear transformation. | ||
* **Range of values**: False | ||
* **Type**: ``boolean`` | ||
* **Default value**: False | ||
* **Required**: *no*. | ||
**Inputs** | ||
* **1**: ``X`` - 2D tensor of type *T* and shape ``[batch_size, input_size]``, input | ||
data. **Required.** | ||
* **2**: ``H_t`` - 2D tensor of type *T* and shape ``[batch_size, hidden_size]``. | ||
Input with initial hidden state data. **Required.** | ||
* **3**: ``W`` - 2D tensor of type *T* and shape ``[3 * hidden_size, input_size]``. | ||
The weights for matrix multiplication, gate order: zrh. **Required.** | ||
* **4**: ``R`` - 2D tensor of type *T* and shape ``[3 * hidden_size, hidden_size]``. | ||
The recurrence weights for matrix multiplication, gate order: zrh. **Required.** | ||
* **5**: ``B`` - 2D tensor of type *T*. The biases. If *linear_before_reset* is set | ||
to ``False``, then the shape is ``[3 * hidden_size]``, gate order: zrh. Otherwise | ||
the shape is ``[4 * hidden_size]`` - the sum of biases for z and r gates (weights and | ||
recurrence weights), the biases for h gate are placed separately. **Required.** | ||
* **6**: ``A`` - 2D tensor of type *T* and shape ``[batch_size, 1]``, the attention | ||
score. **Required.** | ||
**Outputs** | ||
* **1**: ``Ho`` - 2D tensor of type *T* ``[batch_size, hidden_size]``, the last output | ||
value of hidden state. | ||
**Types** | ||
* *T*: any supported floating-point type. | ||
**Example** | ||
.. code-block:: xml | ||
:force: | ||
<layer ... type="AUGRUCell" ...> | ||
<data hidden_size="128"/> | ||
<input> | ||
<port id="0"> <!-- `X` input data --> | ||
<dim>1</dim> | ||
<dim>16</dim> | ||
</port> | ||
<port id="1"> <!-- `H_t` input --> | ||
<dim>1</dim> | ||
<dim>128</dim> | ||
</port> | ||
<port id="3"> <!-- `W` weights input --> | ||
<dim>384</dim> | ||
<dim>16</dim> | ||
</port> | ||
<port id="4"> <!-- `R` recurrence weights input --> | ||
<dim>384</dim> | ||
<dim>128</dim> | ||
</port> | ||
<port id="5"> <!-- `B` bias input --> | ||
<dim>384</dim> | ||
</port> | ||
<port id="6"> <!-- `A` attention score input --> | ||
<dim>1</dim> | ||
<dim>1</dim> | ||
</port> | ||
</input> | ||
<output> | ||
<port id="7"> <!-- `Y` output --> | ||
<dim>1</dim> | ||
<dim>4</dim> | ||
<dim>128</dim> | ||
</port> | ||
<port id="8"> <!-- `Ho` output --> | ||
<dim>1</dim> | ||
<dim>128</dim> | ||
</port> | ||
</output> | ||
</layer> | ||
Oops, something went wrong.