Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

at-views macro to convert a whole block of code to slices=views #20164

Merged
merged 10 commits into from
Jan 24, 2017
3 changes: 3 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,9 @@ This section lists changes that do not have deprecation warnings.
Library improvements
--------------------

* `@views` macro to convert a whole expression or block of code to
use views for all slices ([#20164]).

* `max`, `min`, and related functions (`minmax`, `maximum`, `minimum`,
`extrema`) now return `NaN` for `NaN` arguments ([#12563]).

Expand Down
14 changes: 3 additions & 11 deletions base/broadcast.jl
Original file line number Diff line number Diff line change
Expand Up @@ -503,16 +503,8 @@ end
# explicit calls to view. (All of this can go away if slices
# are changed to generate views by default.)

dotview(args...) = getindex(args...)
dotview(A::AbstractArray, args...) = view(A, args...)
dotview{T<:AbstractArray}(A::AbstractArray{T}, args...) = getindex(A, args...)
# avoid splatting penalty in common cases:
for nargs = 0:5
args = Symbol[Symbol("x",i) for i = 1:nargs]
eval(Expr(:(=), Expr(:call, :dotview, args...),
Expr(:call, :getindex, args...)))
eval(Expr(:(=), Expr(:call, :dotview, :(A::AbstractArray), args...),
Expr(:call, :view, :A, args...)))
end
Base.@propagate_inbounds dotview(args...) = getindex(args...)
Base.@propagate_inbounds dotview(A::AbstractArray, args...) = view(A, args...)
Base.@propagate_inbounds dotview{T<:AbstractArray}(A::AbstractArray{T}, args...) = getindex(A, args...)

end # module
1 change: 1 addition & 0 deletions base/exports.jl
Original file line number Diff line number Diff line change
Expand Up @@ -1384,6 +1384,7 @@ export
@label,
@goto,
@view,
@views,

# SparseArrays module re-exports
SparseArrays,
Expand Down
52 changes: 51 additions & 1 deletion base/subarray.jl
Original file line number Diff line number Diff line change
Expand Up @@ -381,7 +381,8 @@ end
Creates a `SubArray` from an indexing expression. This can only be applied directly to a
reference expression (e.g. `@view A[1,2:end]`), and should *not* be used as the target of
an assignment (e.g. `@view(A[1,2:end]) = ...`).
an assignment (e.g. `@view(A[1,2:end]) = ...`). See also [`@views`](@ref)
to switch an entire block of code to use views for slicing.
"""
macro view(ex)
if isa(ex, Expr) && ex.head == :ref
Expand All @@ -391,3 +392,52 @@ macro view(ex)
throw(ArgumentError("Invalid use of @view macro: argument must be a reference expression A[...]."))
end
end

############################################################################
# @views macro code:

# maybeview is like getindex, but returns a view for slicing operations
# (while remaining equivalent to getindex for scalar indices and non-array types)
@propagate_inbounds maybeview(A, args...) = getindex(A, args...)
@propagate_inbounds maybeview(A::AbstractArray, args...) = view(A, args...)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this need an array-of-arrays special case like dotview has?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, because for scalar indexing it already gives getindex

@propagate_inbounds maybeview(A::AbstractArray, args::Number...) = getindex(A, args...)
@propagate_inbounds maybeview(A) = getindex(A)
@propagate_inbounds maybeview(A::AbstractArray) = getindex(A)

_views(x) = x
_views(x::Symbol) = esc(x)
function _views(ex::Expr)
if ex.head in (:(=), :(.=))
# don't use view on the lhs of an assignment
Expr(ex.head, esc(ex.args[1]), _views(ex.args[2]))
elseif ex.head == :ref
ex = replace_ref_end!(ex)
Expr(:call, :maybeview, _views.(ex.args)...)
else
h = string(ex.head)
if last(h) == '='
# don't use view on the lhs of an op-assignment
Expr(first(h) == '.' ? :(.=) : :(=), esc(ex.args[1]),
Expr(:call, esc(Symbol(h[1:end-1])), _views.(ex.args)...))
else
Expr(ex.head, _views.(ex.args)...)
end
end
end

"""
@views expression
Convert every array-slicing operation in the given expression
(which may be a `begin`/`end` block, loop, function, etc.)
to return a view. Scalar indices, non-array types, and
explicit `getindex` calls (as opposed to `array[...]`) are
unaffected.
Note that the `@views` macro only affects `array[...]` expressions
that appear explicitly in the given `expression`, not array slicing that
occurs in functions called by that code.
"""
macro views(x)
_views(x)
end
4 changes: 3 additions & 1 deletion doc/src/manual/arrays.md
Original file line number Diff line number Diff line change
Expand Up @@ -536,7 +536,9 @@ by copying. A `SubArray` is created with the [`view()`](@ref) function, which is
way as [`getindex()`](@ref) (with an array and a series of index arguments). The result of [`view()`](@ref)
looks the same as the result of [`getindex()`](@ref), except the data is left in place. [`view()`](@ref)
stores the input index vectors in a `SubArray` object, which can later be used to index the original
array indirectly.
array indirectly. By putting the [`@views`](@ref) macro in front of an expression or
block of code, any `array[...]` slice in that expression will be converted to
create a `SubArray` view instead.

`StridedVector` and `StridedMatrix` are convenient aliases defined to make it possible for Julia
to call a wider range of BLAS and LAPACK functions by passing them either [`Array`](@ref) or
Expand Down
37 changes: 37 additions & 0 deletions doc/src/manual/performance-tips.md
Original file line number Diff line number Diff line change
Expand Up @@ -911,6 +911,43 @@ example, but in many contexts it is more convenient to just sprinkle
some dots in your expressions rather than defining a separate function
for each vectorized operation.)

## Consider using views for slices

In Julia, an array "slice" expression like `array[1:5, :]` creates
a copy of that data (except on the left-hand side of an assignment,
where `array[1:5, :] = ...` assigns in-place to that portion of `array`).
If you are doing many operations on the slice, this can be good for
performance because it is more efficient to work with a smaller
contiguous copy than it would be to index into the original array.
On the other hand, if you are just doing a few simple operations on
the slice, the cost of the allocation and copy operations can be
substantial.

An alternative is to create a "view" of the array, which is
an array object (a `SubArray`) that actually references the data
of the original array in-place, without making a copy. (If you
write to a view, it modifies the original array's data as well.)
This can be done for individual slices by calling [`view()`](@ref),
or more simply for a whole expression or block of code by putting
[`@views`](@ref) in front of that expression. For example:

```julia
julia> fcopy(x) = sum(x[2:end-1])

julia> @views fview(x) = sum(x[2:end-1])

julia> x = rand(10^6);

julia> @time fcopy(x);
0.003051 seconds (7 allocations: 7.630 MB)

julia> @time fview(x);
0.001020 seconds (6 allocations: 224 bytes)
```

Notice both the 3× speedup and the decreased memory allocation
of the `fview` version of the function.

## Avoid string interpolation for I/O

When writing data to a file (or other I/O device), forming extra intermediate strings is a source
Expand Down
1 change: 1 addition & 0 deletions doc/src/stdlib/arrays.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ Base.Broadcast.broadcast!
Base.getindex(::AbstractArray, ::Any...)
Base.view
Base.@view
Base.@views
Base.to_indices
Base.Colon
Base.parent
Expand Down
32 changes: 31 additions & 1 deletion test/subarray.jl
Original file line number Diff line number Diff line change
Expand Up @@ -472,7 +472,6 @@ Y = 4:-1:1

@test isa(@view(X[1:3]), SubArray)


@test X[1:end] == @view X[1:end]
@test X[1:end-3] == @view X[1:end-3]
@test X[1:end,2,2] == @view X[1:end,2,2]
Expand All @@ -490,6 +489,37 @@ let size=(x,y)-> error("should not happen")
@test X[1:end,2,2] == @view X[1:end,2,2]
end

# test @views macro
@views let f!(x) = x[1:end-1] .+= x[2:end].^2
x = [1,2,3,4]
f!(x)
@test x == [5,11,19,4]
@test x[1:3] isa SubArray
@test x[2] === 11
@test Dict((1:3) => 4)[1:3] === 4
x[1:2] = 0
@test x == [0,0,19,4]
x[1:2] .= 5:6
@test x == [5,6,19,4]
f!(x[3:end])
@test x == [5,6,35,4]
end
@views @test isa(X[1:3], SubArray)
@test X[1:end] == @views X[1:end]
@test X[1:end-3] == @views X[1:end-3]
@test X[1:end,2,2] == @views X[1:end,2,2]
@test X[1,1:end-2] == @views X[1,1:end-2]
@test X[1,2,1:end-2] == @views X[1,2,1:end-2]
@test X[1,2,Y[2:end]] == @views X[1,2,Y[2:end]]
@test X[1:end,2,Y[2:end]] == @views X[1:end,2,Y[2:end]]
@test X[u...,2:end] == @views X[u...,2:end]
@test X[(1,)...,(2,)...,2:end] == @views X[(1,)...,(2,)...,2:end]

# test macro hygiene
let size=(x,y)-> error("should not happen")
@test X[1:end,2,2] == @views X[1:end,2,2]
end

# issue #18034
# ensure that it is possible to create an isbits, LinearFast view of an immutable Array
let
Expand Down