Skip to content

Commit

Permalink
tidy
Browse files Browse the repository at this point in the history
  • Loading branch information
mcabbott committed Sep 2, 2022
1 parent 2cbddd3 commit e439156
Show file tree
Hide file tree
Showing 2 changed files with 35 additions and 22 deletions.
10 changes: 0 additions & 10 deletions .travis.yml

This file was deleted.

47 changes: 35 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,43 +7,66 @@ into one `AbstractArray`. Given several arrays with the same `eltype`,
or an array of such arrays, it returns a lazy `Stacked{T,N}` view of these:

```julia
lazystack([zeros(2,2), ones(2,2)]) # isa Stacked{Float64, 3, <:Vector{<:Matrix}}
lazystack([1,2,3], 4:6) # isa Stacked{Int, 2, <:Tuple{<:Vector, <:UnitRange}}
julia> lazystack([1:2, 3:4, 5:6])
2×3 lazystack(::Vector{UnitRange{Int64}}) with eltype Int64:
1 3 5
2 4 6

julia> lazystack([pi^ℯ], [ℯ^pi])
1×2 lazystack(::Tuple{Vector{Float64}, Vector{Float64}}) with eltype Float64:
22.4592 23.1407
```

Before v0.1 this function used to be called `stack`, but that name is now exported by Base (from Julia 1.9).
However, `Base.stack` always returns a new dense array, not a lazy container.
Like this package, `Base.stack` makes an array with `size(result) = (size(inner)..., size(outer)...)`.
It always returns a new dense array, not a lazy container.
And instead of two vectors (in the above example) it would want a tuple `stack(([pi^ℯ], [ℯ^pi]))`.

Generators such as `lazystack([i,2i] for i in 1:5)` and arrays of mixed eltype like `lazystack([1,2], [3.0, 4.0], [5im, 6im])` used to be be handled here, and are now simply passed through to `Base.stack`.
Generators such as `lazystack([i,2i] for i in 1:5)` and arrays of mixed eltype like `lazystack([1,2], [3.0, 4.0], [5im, 6im])` used to be be handled here, making a dense array, but are now simply passed through to `Base.stack`.

When the individual slices aren't backed by an Array, as for instance with `CuArray`s on a GPU, then again `Base.stack` is called.
When the individual slices aren't backed by an `Array`, as for instance with `CuArray`s on a GPU, then again `Base.stack` is called.
This should make one big `CuArray`, since scalar indexing of individual slices won't work well.

### Ragged stack

There is also a version which does not demand that slices have equal `size` (or equal `ndims`),
which always returns a new `Array`. You can control the position of slices `using OffsetArrays`:
There is also a version which does not demand that slices have equal `size` (or equal `ndims`).
For now this is not lazy:

```julia
raggedstack([1:n for n in 1:10]) # upper triangular Matrix{Int}
raggedstack(OffsetArray(fill(n,4), rand(-2:2)) for n in 1:10; fill=NaN)
julia> raggedstack([10:10+n for n in 1:3])
4×3 Matrix{Int64}:
10 10 10
11 11 11
0 12 12
0 0 13

julia> using OffsetArrays

julia> raggedstack(OffsetArray(fill(1.0n, 3), rand(-1:1)) for n in 1:10; fill=NaN)
5×10 OffsetArray(::Matrix{Float64}, 0:4, 1:10) with eltype Float64 with indices 0:4×1:10:
NaN 2.0 NaN 4.0 NaN 6.0 7.0 NaN 9.0 NaN
1.0 2.0 3.0 4.0 5.0 6.0 7.0 NaN 9.0 10.0
1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0
1.0 NaN 3.0 NaN 5.0 NaN NaN 8.0 NaN 10.0
NaN NaN NaN NaN NaN NaN NaN 8.0 NaN NaN
```

### Other packages

This one plays well with [OffsetArrays.jl](https://github.com/JuliaArrays/OffsetArrays.jl), and ChainRules-compatible AD such as [Zygote.jl](https://github.com/FluxML/Zygote.jl).
This one plays well with [OffsetArrays.jl](https://github.com/JuliaArrays/OffsetArrays.jl), and [ChainRules.jl](https://github.com/JuliaDiff/ChainRules.jl)-compatible AD such as [Zygote.jl](https://github.com/FluxML/Zygote.jl). It's also used internally by [TensorCast.jl](https://github.com/mcabbott/TensorCast.jl).

Besides which, there are several other ways to achieve similar things:

* For an array of arrays, you can also use [`JuliennedArrays.Align`](https://bramtayl.github.io/JuliennedArrays.jl/latest/#JuliennedArrays.Align). This requires (or enables) you to specify which dimensions of the output belong to the sub-arrays, instead of writing `PermutedDimsArray(stack(...), ...)`.
* There is also [`RecursiveArrayTools.VectorOfArray`](https://github.com/JuliaDiffEq/RecursiveArrayTools.jl#vectorofarray) which as its name hints only allows a one-dimensional container. Linear indexing retreives a slice, not an element, which is sometimes surprising.
* There is also [`RecursiveArrayTools.VectorOfArray`](https://github.com/JuliaDiffEq/RecursiveArrayTools.jl#vectorofarray) which as its name hints only allows a one-dimensional container. (And unlike the package name, nothing is recursive.) Linear indexing retreives a slice, not an element, which is sometimes surprising.
* For a tuple of arrays, [`LazyArrays.Hcat`](https://github.com/JuliaArrays/LazyArrays.jl#concatenation) is at present faster to index than `stack`, but doesn't allow arbitrary dimensions.

And a few more:

* When writing this I missed [`SplitApplyCombine.combinedimsview`](https://github.com/JuliaData/SplitApplyCombine.jl#combinedimsviewarray), which is very similar to `stack`, but doesn't handle tuples.
* Newer than this package is [StackViews.jl](https://github.com/JuliaArrays/StackViews.jl) handles both, with `StackView(A,B,dims=4) == StackView([A,B],4)` creating a 4th dimension; the container is always one-dimensional.
* [`Flux.stack`](https://fluxml.ai/Flux.jl/stable/utilities/#Flux.stack) similarly takes a dimension, but eagerly creates an `Array`.
* Finally, [CatViews.jl](https://github.com/ahwillia/CatViews.jl) offers a lazy `vcat`. But the package is old and I think not so fast.

The lazy inverse:

Expand All @@ -53,7 +76,7 @@ The lazy inverse:

* As does [`PackedVectorsOfVectors`](https://github.com/synchronoustechnologies/PackedVectorsOfVectors.jl), although only 1+1 dimensions. Also has an eager `pack` method which turns a vector of vectors into view of a single larger matrix.

* [`Base.eachslice`](https://docs.julialang.org/en/v1/base/arrays/#Base.eachslice) also views one large array as many slices. This is a generator, but [JuliaLang#32310](https://github.com/JuliaLang/julia/pull/32310) should upgrade it to a multi-dimensional container indexable container.
* [`Base.eachslice`](https://docs.julialang.org/en/v1/base/arrays/#Base.eachslice) also views one large array as many slices. This was a generator, but [JuliaLang#32310](https://github.com/JuliaLang/julia/pull/32310) upgrades it to a multi-dimensional indexable container, in Julia 1.9.

Eager:

Expand Down

0 comments on commit e439156

Please sign in to comment.