Additional tools for type-stable multidimensional coding

[timholy]

This has a couple of goals:

• support a few more convenience methods for CartesianIterator/CartesianRange
• provide better support for writing code without resorting to generated functions

For the 2nd goal (which is the "WIP" part of this), a key step is providing more comprehensive tools for working with tuples. For this reason, I decided to export tail (which we've had in Base for ages), add a corresponding head (which I define locally and have found useful on quite a few occasions), and rewrite ntuple so it doesn't use generated functions. Everything else looks reasonable at the level of @code_native, e.g.,

julia> @code_native head((1,2,3))
        .text
Filename: tuple.jl
Source line: 0
        pushq   %rbp
        movq    %rsp, %rbp
Source line: 21
        movq    (%rsi), %rax
        movq    8(%rsi), %rcx
        movq    %rcx, 8(%rdi)
        movq    %rax, (%rdi)
        movq    %rdi, %rax
        popq    %rbp
        retq

julia> @code_native tail((1,2,3))
        .text
Filename: essentials.jl
Source line: 0
        pushq   %rbp
        movq    %rsp, %rbp
Source line: 64
        movq    8(%rsi), %rax
        movq    16(%rsi), %rcx
        movq    %rcx, 8(%rdi)
        movq    %rax, (%rdi)
        movq    %rdi, %rax
        popq    %rbp
        retq

but my attempted rewrite of ntuple was Not Good:

julia> @code_native ntuple(identity, Val{3})
        .text
Filename: tuple.jl
Source line: 0
        pushq   %rbp
        movq    %rsp, %rbp
        pushq   %r15
        pushq   %r14
        pushq   %r13
        pushq   %r12
        pushq   %rbx
        subq    $88, %rsp
        movq    %rdi, %r14
Source line: 66
        movq    %r14, -120(%rbp)
        leaq    -72(%rbp), %r13
Source line: 32
        movq    $14, -112(%rbp)
        movabsq $jl_tls_states, %rcx
        movq    (%rcx), %rax
        movq    %rax, -104(%rbp)
        leaq    -112(%rbp), %rax
        movq    %rax, (%rcx)
        vxorps  %xmm0, %xmm0, %xmm0
        vmovups %xmm0, -96(%rbp)
        movq    $0, -80(%rbp)
        movq    $0, -56(%rbp)
        movq    $0, -48(%rbp)
        movq    %r14, -72(%rbp)
        movabsq $139726505222272, %r12  # imm = 0x7F149CB5A080
        movq    %r12, -64(%rbp)
        movabsq $jl_apply_generic, %rax
        movl    $2, %esi
        movq    %r13, %rdi
        callq   *%rax
        movq    %rax, -72(%rbp)
        movabsq $jl_f_tuple, %rax
        xorl    %edi, %edi
        movl    $1, %edx
        movq    %r13, %rsi
        callq   *%rax
        movq    %rax, %rbx
        movq    %rbx, -96(%rbp)
Source line: 76
        movq    %rbx, -72(%rbp)
        movabsq $jl_f_nfields, %rax
        xorl    %edi, %edi
Source line: 32
        movl    $1, %edx
Source line: 76
        movq    %r13, %rsi
        callq   *%rax
        movq    (%rax), %r15
Source line: 32
        movq    %rbx, -72(%rbp)
        movq    %r12, -64(%rbp)
        movabsq $jl_f_getfield, %rax
        xorl    %edi, %edi
        movl    $2, %edx
        movq    %r13, %rsi
        callq   *%rax
        movq    %rax, -72(%rbp)
        movq    %r14, -64(%rbp)
        incq    %r15
        movabsq $jl_box_int64, %rax
        movq    %r15, %rdi
        callq   *%rax
        movq    %rax, -56(%rbp)
        movl    $2, %esi
        leaq    -64(%rbp), %r14
        movq    %r14, %rdi
        movabsq $jl_apply_generic, %rax
        callq   *%rax
        movq    %rax, -64(%rbp)
        xorl    %edi, %edi
        movl    $2, %edx
        movq    %r13, %rbx
        movq    %rbx, %rsi
        movabsq $jl_f_tuple, %rax
        callq   *%rax
        movq    %rax, %r13
        movq    %r13, -88(%rbp)
Source line: 76
        movq    %r13, -72(%rbp)
        xorl    %edi, %edi
Source line: 32
        movl    $1, %edx
Source line: 76
        movq    %rbx, %rsi
        movabsq $jl_f_nfields, %rax
        callq   *%rax
        movq    (%rax), %r15
Source line: 32
        movq    %r13, -72(%rbp)
        movq    %r12, -64(%rbp)
        xorl    %edi, %edi
        movl    $2, %edx
        movq    %rbx, %rsi
        movabsq $jl_f_getfield, %rbx
        callq   *%rbx
        movq    %rax, -72(%rbp)
        movq    %r13, -64(%rbp)
        orq     $48, %r12
        movq    %r12, -56(%rbp)
        xorl    %edi, %edi
        movl    $2, %edx
        movq    %r14, %rsi
        callq   *%rbx
        movq    %rax, -64(%rbp)
        movq    -120(%rbp), %rax
        movq    %rax, -56(%rbp)
        incq    %r15
        movq    %r15, %rdi
        movabsq $jl_box_int64, %rax
        callq   *%rax
        movq    %rax, -48(%rbp)
        movl    $2, %esi
        leaq    -56(%rbp), %rdi
        movabsq $jl_apply_generic, %rax
        callq   *%rax
        movq    %rax, -56(%rbp)
        xorl    %edi, %edi
        movl    $3, %edx
Source line: 66
        leaq    -72(%rbp), %rsi
Source line: 32
        movabsq $jl_f_tuple, %rax
        callq   *%rax
        movq    %rax, -80(%rbp)
Source line: 73
        movq    -104(%rbp), %rcx
Source line: 32
        movabsq $jl_tls_states, %rdx
Source line: 73
        movq    %rcx, (%rdx)
        addq    $88, %rsp
        popq    %rbx
        popq    %r12
        popq    %r13
        popq    %r14
        popq    %r15
        popq    %rbp
        retq

Compare master (which uses a generated function, though for N=3 we don't call it):

julia> @code_native ntuple(identity, Val{3})
        .text
Filename: tuple.jl
Source line: 0
        pushq   %rbp
        movq    %rsp, %rbp
Source line: 56
        movq    $3, 16(%rdi)
        movq    $2, 8(%rdi)
        movq    $1, (%rdi)
        movq    %rdi, %rax
        popq    %rbp
        retq

I presume that #13359 strikes again?

If it's the best choice for now, I can drop the rewrite of ntuple, but I thought I'd throw it out there in case anyone has ideas.

[StefanKarpinski]

If we're going to have head and tail, imo, they should be generic functions that work for other ordered collections too. Or even better, we use the same vocabulary to manipulate tuples as we do for vectors. I guess that means that head(t) would be spelled t[1] and tail(t) would be spelled t[2:end].

[JeffBezanson]

t[2:end] is not a good way to express this since it doesn't work well for non-indexable collections like linked lists, and is much harder to statically analyze.

[JeffBezanson]

Note that we also have rest, but it is lazy and we could decide to just make it always lazy and have tail be eager.

[nalimilan]

head and tail sound quite restricted if all they do is skip the first/last element. One possible approach is to follow R and allow for an additional argument giving the number of elements to retain, with negative values giving the number of elements to skip. This could also be merged with first and last.

[JeffBezanson]

Good point about first. It's identical to head afaict.

Yes, a second argument to tail to get the nth tail is a standard extension. Interestingly, that is distinct from rest which takes a state argument. So maybe rest should be called something like startingfrom.

The standard extension to head is to give the first n elements.

[simonster]

If you give head/tail a second argument, then I don't think you'll be able to infer a concrete return type for tuples without an inference hack.

[timholy]

Sorry, I should have stated that head(t) == t[1:end-1]. We don't currently have a good type-stable way of peeling off the last element, in the same way that tail peels off the first element. The main use I've found for these is in constructs like

A = Array(eltype(B), head(size(B)))

which would make an array of size size(B)[1:end-1] in a way that julia can infer the dimensionality of A.

head(t, n::Integer) would not be type-stable, but head(t, ::Type{Val{n}}) could be.

I seem to remember that Jeff once posted a gist containing examples of implementing arithmetic at compile-time, and this PR is kind of aiming in that general direction.

[eschnett]

head usually returns a single element. Would you like to call you function e.g. init instead?

[timholy]

Didn't know that. I'm fine with whatever name seems sensible.

[tkelman]

most ?

[timholy]

I was going to suggest head_and_shoulders_and_knees_but_not_toes. Probably too many underscores for Jeff, though 😛.

I like most, and perhaps I'm even more fond of front. But before we go too whole-hog on names, the bigger decision is whether this is useful enough to have as an exported function. Given @StefanKarpinski's point about generic code, one wonders whether these kinds of things should be in a Base.Tuples module, or perhaps a package. Perhaps the only real argument for having such things in Base is the fact that this feels like low-level plumbing.

For reference: there are relatively few calls to tail in Base, but it is defined in essentials so it is nevertheless important. head/init/most/front is probably less useful than tail.

[timholy]

One place where such manipulations are currently ugly: this impenetrable function (called from, e.g., here) could be written more clearly as

tail(fill_to_length(size(Rval), 1, size(A)))

where fill_to_length(out::Tuple, val, ref::Tuple) appends val to the tuple out until it has the same length as the tuple ref. (Unless you use Val{N}, you need a tuple-ref if you want this to be type-stable.)

These are the kinds of manipulations I frequently need to make when writing code for arbitrary dimensions and inputs whose dimensionality may not match.

[timholy]

I've stripped out any ambiguous/problematic stuff (the ntuple rewrite, head/front). Will merge when this passes CI.