Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No positional Integer selection possible in time #400

Open
Sonicious opened this issue May 29, 2024 · 3 comments
Open

No positional Integer selection possible in time #400

Sonicious opened this issue May 29, 2024 · 3 comments
Labels
breaking bug Something isn't working help wanted Extra attention is needed

Comments

@Sonicious
Copy link
Collaborator

I want to get several time slices from a cube lazily. Using the Tutorial, I assumed the following will work:

path="gs://cmip6/CMIP6/ScenarioMIP/DKRZ/MPI-ESM1-2-HR/ssp585/r1i1p1f1/3hr/tas/gn/v20190710/"
store = zopen(path, consolidated=true)
ds = open_dataset(store)

ds[Ti = 5] # works
ds[Ti = 1:5] # ERROR: ArgumentError: Chunk sizes must be strictly positive

ds[lat = 1] # works
ds[lat = 1:5] # works

Is it not intended to use integer selection on the Ti axis?

@Sonicious Sonicious changed the title No positional Integer selection possible No positional Integer selection possible in time May 29, 2024
@felixcremer
Copy link
Member

I'd say this should work. Could you copy the full error? Does of work with Interval access?

@Sonicious
Copy link
Collaborator Author

Here is the full error:

julia> show(err)
1-element ExceptionStack:
ArgumentError: Chunk sizes must be strictly positive
Stacktrace:
  [1] RegularChunks
    @ C:\Users\marti\.julia\packages\DiskArrays\mGRbQ\src\chunks.jl:26 [inlined]
  [2] subsetchunks(r::DiskArrays.RegularChunks, subs::UnitRange{Int64})
    @ DiskArrays C:\Users\marti\.julia\packages\DiskArrays\mGRbQ\src\chunks.jl:50
  [3] #104
    @ .\array.jl:0 [inlined]
  [4] iterate
    @ .\generator.jl:47 [inlined]
  [5] grow_to!(dest::Vector{DiskArrays.RegularChunks}, itr::Base.Generator{Base.Iterators.Filter{DiskArrays.var"#105#108"{Tuple{}}, UnitRange{Int64}}, DiskArrays.var"#104#107"{DiskArrays.GridChunks{3, Tuple{DiskArrays.RegularChunks, DiskArrays.RegularChunks, DiskArrays.RegularChunks}}, Tuple{Base.Slice{Base.OneTo{Int64}}, Base.Slice{Base.OneTo{Int64}}, UnitRange{Int64}}}}, st::Int64)
    @ Base .\array.jl:938
  [6] grow_to!(dest::Vector{DiskArrays.RegularChunks}, itr::Base.Generator{Base.Iterators.Filter{DiskArrays.var"#105#108"{Tuple{}}, UnitRange{Int64}}, DiskArrays.var"#104#107"{DiskArrays.GridChunks{3, Tuple{DiskArrays.RegularChunks, DiskArrays.RegularChunks, DiskArrays.RegularChunks}}, Tuple{Base.Slice{Base.OneTo{Int64}}, Base.Slice{Base.OneTo{Int64}}, UnitRange{Int64}}}})
    @ Base .\array.jl:911
  [7] collect
    @ .\array.jl:831 [inlined]
  [8] eachchunk_view
    @ C:\Users\marti\.julia\packages\DiskArrays\mGRbQ\src\subarray.jl:33 [inlined]
  [9] eachchunk
    @ C:\Users\marti\.julia\packages\DiskArrays\mGRbQ\src\subarray.jl:25 [inlined]
 [10] rebuild(A::YAXArray{Float32, 3, ZArray{Float32, 3, Zarr.BloscCompressor, GCStore}, Tuple{Dim{:lon, DimensionalData.Dimensions.Lookups.Sampled{Float64, StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, DimensionalData.Dimensions.Lookups.ForwardOrdered, DimensionalData.Dimensions.Lookups.Regular{Float64}, DimensionalData.Dimensions.Lookups.Points, DimensionalData.Dimensions.Lookups.NoMetadata}}, Dim{:lat, DimensionalData.Dimensions.Lookups.Sampled{Float64, Vector{Float64}, DimensionalData.Dimensions.Lookups.ForwardOrdered, DimensionalData.Dimensions.Lookups.Irregular{Tuple{Nothing, Nothing}}, DimensionalData.Dimensions.Lookups.Points, DimensionalData.Dimensions.Lookups.NoMetadata}}, DimensionalData.Dimensions.Ti{DimensionalData.Dimensions.Lookups.Sampled{DateTime, Vector{DateTime}, DimensionalData.Dimensions.Lookups.ForwardOrdered, DimensionalData.Dimensions.Lookups.Irregular{Tuple{Nothing, Nothing}}, DimensionalData.Dimensions.Lookups.Points, DimensionalData.Dimensions.Lookups.NoMetadata}}}, Dict{String, Any}}, data::DiskArrays.SubDiskArray{Float32, 3, ZArray{Float32, 3, Zarr.BloscCompressor, GCStore}, Tuple{Base.Slice{Base.OneTo{Int64}}, Base.Slice{Base.OneTo{Int64}}, UnitRange{Int64}}, false}, dims::Tuple{Dim{:lon, DimensionalData.Dimensions.Lookups.Sampled{Float64, StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, DimensionalData.Dimensions.Lookups.ForwardOrdered, DimensionalData.Dimensions.Lookups.Regular{Float64}, DimensionalData.Dimensions.Lookups.Points, DimensionalData.Dimensions.Lookups.NoMetadata}}, Dim{:lat, DimensionalData.Dimensions.Lookups.Sampled{Float64, SubArray{Float64, 1, Vector{Float64}, Tuple{Base.Slice{Base.OneTo{Int64}}}, true}, DimensionalData.Dimensions.Lookups.ForwardOrdered, DimensionalData.Dimensions.Lookups.Irregular{Tuple{Float64, Float64}}, DimensionalData.Dimensions.Lookups.Points, DimensionalData.Dimensions.Lookups.NoMetadata}}, DimensionalData.Dimensions.Ti{DimensionalData.Dimensions.Lookups.Sampled{DateTime, SubArray{DateTime, 1, Vector{DateTime}, Tuple{UnitRange{Int64}}, true}, DimensionalData.Dimensions.Lookups.ForwardOrdered, DimensionalData.Dimensions.Lookups.Irregular{Tuple{Nothing, Nothing}}, DimensionalData.Dimensions.Lookups.Points, DimensionalData.Dimensions.Lookups.NoMetadata}}}, refdims::Tuple{}, name::DimensionalData.NoName, metadata::Dict{String, Any})
    @ YAXArrays.Cubes C:\Users\marti\.julia\packages\YAXArrays\jjg2q\src\Cubes\Cubes.jl:200
 [11] rebuild
    @ C:\Users\marti\.julia\packages\DimensionalData\BoJag\src\array\array.jl:85 [inlined]
 [12] rebuildsliced
    @ C:\Users\marti\.julia\packages\DimensionalData\BoJag\src\array\array.jl:100 [inlined]
 [13] rebuildsliced
    @ C:\Users\marti\.julia\packages\DimensionalData\BoJag\src\array\array.jl:99 [inlined]
 [14] view
    @ C:\Users\marti\.julia\packages\DimensionalData\BoJag\src\array\indexing.jl:125 [inlined]
 [15] _dim_view
    @ C:\Users\marti\.julia\packages\DimensionalData\BoJag\src\array\indexing.jl:110 [inlined]
 [16] #view#110
    @ C:\Users\marti\.julia\packages\DimensionalData\BoJag\src\array\indexing.jl:81 [inlined]
 [17] getindex(::YAXArray{Float32, 3, ZArray{Float32, 3, Zarr.BloscCompressor, GCStore}, Tuple{Dim{:lon, DimensionalData.Dimensions.Lookups.Sampled{Float64, StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, DimensionalData.Dimensions.Lookups.ForwardOrdered, DimensionalData.Dimensions.Lookups.Regular{Float64}, DimensionalData.Dimensions.Lookups.Points, DimensionalData.Dimensions.Lookups.NoMetadata}}, Dim{:lat, DimensionalData.Dimensions.Lookups.Sampled{Float64, Vector{Float64}, DimensionalData.Dimensions.Lookups.ForwardOrdered, DimensionalData.Dimensions.Lookups.Irregular{Tuple{Nothing, Nothing}}, DimensionalData.Dimensions.Lookups.Points, DimensionalData.Dimensions.Lookups.NoMetadata}}, DimensionalData.Dimensions.Ti{DimensionalData.Dimensions.Lookups.Sampled{DateTime, Vector{DateTime}, DimensionalData.Dimensions.Lookups.ForwardOrdered, DimensionalData.Dimensions.Lookups.Irregular{Tuple{Nothing, Nothing}}, DimensionalData.Dimensions.Lookups.Points, DimensionalData.Dimensions.Lookups.NoMetadata}}}, Dict{String, Any}}; kwargs::@Kwargs{Ti::UnitRange{Int64}})
    @ YAXArrays.Cubes C:\Users\marti\.julia\packages\YAXArrays\jjg2q\src\Cubes\Cubes.jl:487
 [18] subsetifdimexists(a::YAXArray{Float32, 3, ZArray{Float32, 3, Zarr.BloscCompressor, GCStore}, Tuple{Dim{:lon, DimensionalData.Dimensions.Lookups.Sampled{Float64, StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, DimensionalData.Dimensions.Lookups.ForwardOrdered, DimensionalData.Dimensions.Lookups.Regular{Float64}, DimensionalData.Dimensions.Lookups.Points, DimensionalData.Dimensions.Lookups.NoMetadata}}, Dim{:lat, DimensionalData.Dimensions.Lookups.Sampled{Float64, Vector{Float64}, DimensionalData.Dimensions.Lookups.ForwardOrdered, DimensionalData.Dimensions.Lookups.Irregular{Tuple{Nothing, Nothing}}, DimensionalData.Dimensions.Lookups.Points, DimensionalData.Dimensions.Lookups.NoMetadata}}, DimensionalData.Dimensions.Ti{DimensionalData.Dimensions.Lookups.Sampled{DateTime, Vector{DateTime}, DimensionalData.Dimensions.Lookups.ForwardOrdered, DimensionalData.Dimensions.Lookups.Irregular{Tuple{Nothing, Nothing}}, DimensionalData.Dimensions.Lookups.Points, DimensionalData.Dimensions.Lookups.NoMetadata}}}, Dict{String, Any}}; kwargs::@Kwargs{Ti::UnitRange{Int64}})
    @ YAXArrays.Datasets C:\Users\marti\.julia\packages\YAXArrays\jjg2q\src\DatasetAPI\Datasets.jl:178
 [19] (::YAXArrays.Datasets.var"#32#34"{@Kwargs{Ti::UnitRange{Int64}}})(ds::Symbol)
    @ YAXArrays.Datasets C:\Users\marti\.julia\packages\YAXArrays\jjg2q\src\DatasetAPI\Datasets.jl:187
 [20] iterate
    @ .\generator.jl:47 [inlined]
 [21] collect_to!(dest::Vector{Pair{Symbol, YAXArray{Float64, 0, ZArray{Float64, 0, Zarr.NoCompressor, GCStore}, Tuple{}, Dict{String, Any}}}}, itr::Base.Generator{Vector{Symbol}, YAXArrays.Datasets.var"#32#34"{@Kwargs{Ti::UnitRange{Int64}}}}, offs::Int64, st::Int64)
    @ Base .\array.jl:892
 [22] collect_to_with_first!(dest::Vector{Pair{Symbol, YAXArray{Float64, 0, ZArray{Float64, 0, Zarr.NoCompressor, GCStore}, Tuple{}, Dict{String, Any}}}}, v1::Pair{Symbol, YAXArray{Float64, 0, ZArray{Float64, 0, Zarr.NoCompressor, GCStore}, Tuple{}, Dict{String, Any}}},
 itr::Base.Generator{Vector{Symbol}, YAXArrays.Datasets.var"#32#34"{@Kwargs{Ti::UnitRange{Int64}}}}, st::Int64)
    @ Base .\array.jl:870
 [23] _collect(c::Vector{Symbol}, itr::Base.Generator{Vector{Symbol}, YAXArrays.Datasets.var"#32#34"{@Kwargs{Ti::UnitRange{Int64}}}}, ::Base.EltypeUnknown, isz::Base.HasShape{1})
    @ Base .\array.jl:864
 [24] collect_similar(cont::Vector{Symbol}, itr::Base.Generator{Vector{Symbol}, YAXArrays.Datasets.var"#32#34"{@Kwargs{Ti::UnitRange{Int64}}}})
    @ Base .\array.jl:763
 [25] map(f::Function, A::Vector{Symbol})
    @ Base .\abstractarray.jl:3285
 [26] getindex(x::Dataset; var::Nothing, kwargs::@Kwargs{Ti::UnitRange{Int64}})
    @ YAXArrays.Datasets C:\Users\marti\.julia\packages\YAXArrays\jjg2q\src\DatasetAPI\Datasets.jl:187
 [27] top-level scope
    @ REPL[10]:1
julia>

@meggart
Copy link
Member

meggart commented May 30, 2024

This is an old remaining feature from times when we did not depend on DD yet and integer indexing was not available in YAXArrays. In this case integers are interpreted as years to allow things like cube[time=2001:2020]. However, now that we allow integer indexing we should probably remove these lines

if d isa DD.Ti
if v isa UnitRange{Int}
v = Date(first(v))..Date(last(v),12,31)
end
d2[:Ti] = v
and register a breaking change since quite some tutorials used this time selection feature.

@lazarusA lazarusA added bug Something isn't working help wanted Extra attention is needed breaking labels Sep 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking bug Something isn't working help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

4 participants