Implement Einsum reference in nGraph interpreter

[rkazants]

Description:

1. It implements Einsum-7 reference. The implementation is added into nGraph interpreter due to huge size of the implementation. The interpreter binary size (ngraph_reference lib) increases from 2414KB to 3292KB by 878KB (or 36%). Since plugins do not support Einsum-7 operation directly, there exists EinsumDecomposition transformation that helps to complete all model requests.
2. This Einsum reference does not have limitations and supports all equations including repeated labels in input subscribt (aka diagonal extraction), ellipsis, and others.
3. The reference was tested using 11 unit tests with simple input data and ones generated with numpy

Ticket: 49327

Details: The main idea was inherited from EinsumDecomposition transformation implementation in PR5529 and extended to support diagonal extraction and ellipsis in an equation. To get the idea about support of equations without diagonal extraction and without ellipsis, please refer to PR5529 details where it describes the procedure to split labels into three groups (common, separated, reduced dimensions), permute them for grouping, apply MatMul operation, and reshape back the resulted tensor with transpose to get layout adjusted with output subscript.
Here it describes two cases: diagonal extraction and ellipsis.
EinsumDiagonal. For example, Einsum with equation="abb->a" will extract diagonals per batch (i.e. zero dimension):

It makes sense to note that diagonal extraction must be performed in the first step for each operand in case repeated labels in the corresponding input subscripts. For example, diagonal must be extracted for the first operand of Einsum with equation="abbac,bd->ad" in the first step. Only after the first step you can proceed with handling ellipsis and details from PR5529.
Let us perform diagonal extraction for the first operand with input subscript abbac and shape equal to [2, 3, 3, 2, 4].

1. For this, determine repeated labels and corresponding dimension indices for them. In this case, a and b are repeated labels and [0, 3] and [1, 2] are the corresponding dimension indices.
2. Build identity tensors for each repeated label. For a label, it will be I1 of shape [2,1,1,2,1] where I1[k,*,*,k,*]=1 for k=0,1 and 0 - otherwise. For b label, it will be I2 of shape [1,3,3,1,1] where I2[k,*,*,k,*]=1 for k=0,1,2 and 0 - otherwise. Note that identity rank matches to the input rank and its dimension sizes equal dimension sizes corresponding to given label from input shape and equal to 1 for all other labels.
3. Multiples all identities by element-wise operation I=I1*I2*...*In where n is a number of repeated labels.
4. Multiples the first operand by I and perform reduce-sum along dimensions from the corresponding dimension indices for repeated labels excluding the first dimension, i.e. [3] for a and [2] for b. Hence, the extracted diagonal will be of shape [2,3,4] with subscript abc.

Ellipsis. In case of ellipsis, the procedure described in PR5529 is extended with an ellipsis case. All steps from PR5529 are modified so that it takes into account that one ellipsis label can cover multiple dimension. Also, before MatMul operation you need to broadcast operands if they both have dimensions covered by ellipsis label.

Signed-off-by: Roman Kazantsev [email protected]

[lazarevevgeny]

Please, add information about the binary size increase of the ngraph library because of this operation.

[ilyachur]

Could you set the milestone for this activity?
Does this PR fix some ticket?

[rkazants]

Could you set the milestone for this activity?
Does this PR fix some ticket?

Done.

[rkazants]

Please, add information about the binary size increase of the ngraph library because of this operation.

Done

[lazarevevgeny]

No major comments.