The transpose function is problematic

[rgommers]

There are multiple problems, both related to adoption due to conflicting definitions, and because NumPy's transpose does something no one really wants (namely, swap all axes).

The current array API transpose may not be correct:

• https://data-apis.org/array-api/latest/API_specification/linear_algebra_functions.html#transpose-x-axes-none says "swap all axes"
• Linear Algebra design overview #147 claims the intent is to only swap the last two axes, to be consistent with all other linalg functions.

PyTorch names the numpy.transpose equivalent permute, and PyTorch's transpose is np.swapaxes. Relevant issues PyTorch issues:

• torch.transpose is divergent from np.transpose pytorch/pytorch#50275
• [ux] Proposal to have t() === transpose(-1, -2), since batches are very frequent pytorch/pytorch#51280

Relevant NumPy issues:

• Feature request: T method for transposition applying to 1D vectors numpy/numpy#9530
• ENH: ndarray.T2 for 2D transpose. numpy/numpy#7495
• Hermitian Transpose Syntax numpy/numpy#13797 (note that there's np.linalg.linalg.transpose which is private but does have the desired behavior)

There's multiple ways of dealing with this problem:

1. Accept the status quo
2. Remove transpose completely, and use permute or some other function name with the behavior we want by default (do the right thing for batches of 2-D matrices)
3. Add both, by adding a matrix_transpose function and a corresponding .mT (or .MT) attribute

I searched the JAX and TensorFlow issue trackers as well and, somewhat surprisingly, didn't find an issue about this topic.

[shoyer]

I think separating the functionality of np.transpose() into two separate functions/methods would make sense. As you note, there are two distinct use cases:

1. Swap the last two dimensions
2. Permute axes

The only real relation is that (1) can be implemented via (2).

From my perspective, the functionality (1) would be the best fit for the name "transpose", and (2) could use some other name, like "permute axes".

[kgryte]

During the consortium meeting on 2021-07-14, the following was considered a path forward:

1. Limit the .T attribute on array instances to 2D. If an array instance has more than two dimensions is not two-dimensional, an error should be raised. This allows retaining the current behavior for .T for the primary use case of taking the transpose of a matrix, while removing the less desired behavior of reversing all axes. PR: gh-245.
2. Introduce a .mT attribute on array instances which performs a batched matrix transpose for the last two dimensions. PR: gh-246.
3. Remove the transpose function from the specification and replace with a permute API. PR: gh-247.
4. Introduce a matrix_transpose function for performing a batched matrix transpose. PR: gh-248.

[shoyer]

• Remove the transpose function from the specification and replace with a permute API. PR: gh-247.

any thoughts on the name permute vs permute_axes?

I like permute_axes a little bit better, because I can imagine "permute" alternatively referring to array elements.

[rgommers]

Good point @shoyer, there's also https://numpy.org/devdocs/reference/random/generated/numpy.random.Generator.permuted.html which is quite similar. I don't have a strong preference, but permute_axes is indeed slightly clearer.

[kgryte]

Some other reference points:

• MATLAB uses permute.
• Julia uses permutedims
• PyTorch uses permute

I am fine with either permute or permute_axes. Although, maybe we should be consistent with names. For example,

• expand_dims, which is currently included in the specification.
• while not currently included in the specification, swapaxes.

In which case, to throw another hat in the ring, maybe permute_dims, if we want to be more explicit about what is being permuted?

[rgommers]

permute_dims seems quite reasonable too. If we consistently name keywords axis/axes, using _dims for function names and ndim for attribute is at least some kind of consistency.

[kgryte]

As gh-245, gh-246, gh-247, and gh-248 have all been merged, closing this out.