Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fill_value should provide default value #79

Closed
tbenst opened this issue Jan 10, 2022 · 1 comment · Fixed by #84
Closed

fill_value should provide default value #79

tbenst opened this issue Jan 10, 2022 · 1 comment · Fixed by #84

Comments

@tbenst
Copy link

tbenst commented Jan 10, 2022

From Zarr spec,

fill_value
A scalar value providing the default value to use for uninitialized portions of the array, or null if no fill_value is to be used.

I think that fill_value = null would correspond to missing in Julia.

Python Zarr also has the following example:

>>> import zarr
>>> z = zarr.full((10000, 10000), chunks=(1000, 1000), fill_value=42)
>>> z
<zarr.core.Array (10000, 10000) float64>
>>> z[:2, :2]
array([[42., 42.],
       [42., 42.]])

Modifying slightly, we can save to disk with z1 = zarr.open_array('/tmp/example.zarr', mode='w', shape=(10,10),fill_value=42). When we open with Zarr.jl,

julia> z1 = zopen("/tmp/example.zarr")
ZArray{Union{Missing, Float64}} of size 10 x 10

julia> z1[1:2,1:2]
2×2 reshape(::Matrix{Union{Missing, Float64}}, 2, 2) with eltype Union{Missing, Float64}:
 missing  missing
 missing  missing

Per spec & python compatibility, this should be [42 42; 42 42]. I'm not sure how hard this would be to change, but I'm happy to look into it if a PR would be welcome? I think this would also resolve #39

@meggart
Copy link
Collaborator

meggart commented Jan 18, 2022

Yeah that was a misinterpretation of the specs from my side, where I assumed that FillValues should always represent missing values. I think the fix will be simple, but I would really like to keep an option for the old behavior as a kwarg through something like fillvalueasmissing = false. because using FillValue to encode missing values is a very common use case. Of course this will be breaking, so would be good to fix in conjunction with #78 in case we make a breaking change there as well and then bump the minor version of the package.

I will prepare a fix soon.

@meggart meggart mentioned this issue Mar 3, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants