Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

xarray.open_mzar: open multiple zarr files (in parallel) #4003

Closed
wants to merge 58 commits into from
Closed

xarray.open_mzar: open multiple zarr files (in parallel) #4003

wants to merge 58 commits into from

Commits on Apr 23, 2020

  1. create def for multiple zarr files and added commentary/definition, w… 

    …hich matches almost exactly that of ``xr.open_mfdatasets``, but withou ``engine``
    Miguel Jimenez-Urias committed Apr 23, 2020
    Configuration menu
    Copy the full SHA
    f55ed1c View commit details
    Browse the repository at this point in the history

  2. just as with xr.open_mfdatasets, identify the paths as local dire… 

    …ctory paths/strings
    Miguel Jimenez-Urias committed Apr 23, 2020
    Configuration menu
    Copy the full SHA
    49f6512 View commit details
    Browse the repository at this point in the history

  3. added error if no path

    Miguel Jimenez-Urias committed Apr 23, 2020
    Configuration menu
    Copy the full SHA
    f35a3e5 View commit details
    Browse the repository at this point in the history

  4. finished copying similar code from xr.open_mfdatasets

    Miguel Jimenez-Urias committed Apr 23, 2020
    Configuration menu
    Copy the full SHA
    9f728aa View commit details
    Browse the repository at this point in the history

  5. remove blank lines

    Miguel Jimenez-Urias committed Apr 23, 2020
    Configuration menu
    Copy the full SHA
    8d0a844 View commit details
    Browse the repository at this point in the history

  6. fixed typo

    Miguel Jimenez-Urias committed Apr 23, 2020
    Configuration menu
    Copy the full SHA
    b3b0f1d View commit details
    Browse the repository at this point in the history

  7. added xr.open_mzarr() to the list of available modules to call

    Miguel Jimenez-Urias committed Apr 23, 2020
    Configuration menu
    Copy the full SHA
    2221943 View commit details
    Browse the repository at this point in the history

  8. imported missing function

    Miguel Jimenez-Urias committed Apr 23, 2020
    Configuration menu
    Copy the full SHA
    ac35e7c View commit details
    Browse the repository at this point in the history

  9. imported missing glob

    Miguel Jimenez-Urias committed Apr 23, 2020
    Configuration menu
    Copy the full SHA
    64654f3 View commit details
    Browse the repository at this point in the history

  10. imported function from backend.api

    Miguel Jimenez-Urias committed Apr 23, 2020
    Configuration menu
    Copy the full SHA
    d5a5cef View commit details
    Browse the repository at this point in the history

  11. imported function to facilitate mzarr

    Miguel Jimenez-Urias committed Apr 23, 2020
    Configuration menu
    Copy the full SHA
    4c0ef19 View commit details
    Browse the repository at this point in the history

  12. correctly imported functions from core to mzarr

    Miguel Jimenez-Urias committed Apr 23, 2020
    Configuration menu
    Copy the full SHA
    d158c21 View commit details
    Browse the repository at this point in the history

  13. imported to use on open_mzarr

    Miguel Jimenez-Urias committed Apr 23, 2020
    Configuration menu
    Copy the full SHA
    5171420 View commit details
    Browse the repository at this point in the history

  14. removed lock and autoclose since not taken by open_zarr

    Miguel Jimenez-Urias committed Apr 23, 2020
    Configuration menu
    Copy the full SHA
    e1e51bb View commit details
    Browse the repository at this point in the history

  15. fixed typo

    Miguel Jimenez-Urias committed Apr 23, 2020
    Configuration menu
    Copy the full SHA
    b6bf2cf View commit details
    Browse the repository at this point in the history

  16. class is not needed since zarr stores don`t remain open

    Miguel Jimenez-Urias committed Apr 23, 2020
    Configuration menu
    Copy the full SHA
    3bc4be8 View commit details
    Browse the repository at this point in the history

  17. removed old behavior

    Miguel Jimenez-Urias committed Apr 23, 2020
    Configuration menu
    Copy the full SHA
    a79b125 View commit details
    Browse the repository at this point in the history

  18. set default

    Miguel Jimenez-Urias committed Apr 23, 2020
    Configuration menu
    Copy the full SHA
    2d3bbb5 View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2020

  1. listed open_mzarr

    Miguel Jimenez-Urias committed Apr 25, 2020
    Configuration menu
    Copy the full SHA
    f7cf580 View commit details
    Browse the repository at this point in the history

  2. removed unused imported function

    Miguel Jimenez-Urias committed Apr 25, 2020
    Configuration menu
    Copy the full SHA
    53c8623 View commit details
    Browse the repository at this point in the history

  3. imported Path - hadn`t before

    Miguel Jimenez-Urias committed Apr 25, 2020
    Configuration menu
    Copy the full SHA
    34d755e View commit details
    Browse the repository at this point in the history

  4. remove unncessesary comments

    Miguel Jimenez-Urias committed Apr 25, 2020
    Configuration menu
    Copy the full SHA
    b39b37e View commit details
    Browse the repository at this point in the history

  5. modified comments

    Miguel Jimenez-Urias committed Apr 25, 2020
    Configuration menu
    Copy the full SHA
    276006a View commit details
    Browse the repository at this point in the history

  6. isorted zarr

    Miguel Jimenez-Urias committed Apr 25, 2020
    Configuration menu
    Copy the full SHA
    6f04be6 View commit details
    Browse the repository at this point in the history

  7. isorted

    Miguel Jimenez-Urias committed Apr 25, 2020
    Configuration menu
    Copy the full SHA
    aa97e1a View commit details
    Browse the repository at this point in the history

Commits on Apr 28, 2020

  1. erased open_mzarr. Added capability to open_dataset to open zarr files

    Miguel Jimenez-Urias committed Apr 28, 2020
    Configuration menu
    Copy the full SHA
    06de16a View commit details
    Browse the repository at this point in the history

  2. removed imported but unused

    Miguel Jimenez-Urias committed Apr 28, 2020
    Configuration menu
    Copy the full SHA
    f94fc9f View commit details
    Browse the repository at this point in the history

  3. comment to zarr engine

    Miguel Jimenez-Urias committed Apr 28, 2020
    Configuration menu
    Copy the full SHA
    16e08e3 View commit details
    Browse the repository at this point in the history

  4. added chunking code from open_zarr

    Miguel Jimenez-Urias committed Apr 28, 2020
    Configuration menu
    Copy the full SHA
    22828fc View commit details
    Browse the repository at this point in the history

  5. remove import open_mzarr`

    Miguel Jimenez-Urias committed Apr 28, 2020
    Configuration menu
    Copy the full SHA
    021f2cc View commit details
    Browse the repository at this point in the history

  6. removed open_mzarr` from top-level-function

    Miguel Jimenez-Urias committed Apr 28, 2020
    Configuration menu
    Copy the full SHA
    985f28c View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2020

  1. missing return in nested function

    Miguel Jimenez-Urias committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    e8ed887 View commit details
    Browse the repository at this point in the history

  2. moved outside of nested function, had touble with reading before assi… 

    …gnement
    Miguel Jimenez-Urias committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    d693514 View commit details
    Browse the repository at this point in the history

  3. added missing argument associated with zarr stores, onto the definiti… 

    …on of open_dataset
    Miguel Jimenez-Urias committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    df34f18 View commit details
    Browse the repository at this point in the history

  4. isort zarr.py

    Miguel Jimenez-Urias committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    98351c7 View commit details
    Browse the repository at this point in the history

  5. removed blank lines, fixed typo on chunks

    Miguel Jimenez-Urias committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    160bd67 View commit details
    Browse the repository at this point in the history

  6. removed imported but unused

    Miguel Jimenez-Urias committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    7e57e9b View commit details
    Browse the repository at this point in the history

  7. restored conditional for auto

    Miguel Jimenez-Urias committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    ac0f093 View commit details
    Browse the repository at this point in the history

  8. removed imported but unused dask.array

    Miguel Jimenez-Urias committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    6a1516c View commit details
    Browse the repository at this point in the history

  9. added capabilities for file_or_obj to be a mutablemapper such as `fss… 

    …pec.get_mapper`, and thus compatible with `intake-xarray`
    Miguel Jimenez-Urias committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    8999faf View commit details
    Browse the repository at this point in the history

  10. moved to a different conditional since file_or_obj is a mutablemappin… 

    …g, not a str, path or AbstractDataStore
    Miguel Jimenez-Urias committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    5df0985 View commit details
    Browse the repository at this point in the history

  11. isort api.py

    Miguel Jimenez-Urias committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    2d94ea2 View commit details
    Browse the repository at this point in the history

  12. restored the option for when file_or_obk is a str, such as an url.

    Miguel Jimenez-Urias committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    377ef53 View commit details
    Browse the repository at this point in the history

  13. fixed relabel

    Miguel Jimenez-Urias committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    f48c84b View commit details
    Browse the repository at this point in the history

  14. update open_dataset for zarr files

    Miguel Jimenez-Urias committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    8376cca View commit details
    Browse the repository at this point in the history

  15. remove open_zarr from tests, now open_dataset(engine=zarr)

    Miguel Jimenez-Urias committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    aed1cc5 View commit details
    Browse the repository at this point in the history

  16. remove extra file, and raise deprecating warning on open_zarr

    Miguel Jimenez-Urias committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    b488363 View commit details
    Browse the repository at this point in the history

  17. added internal call to open_dataset from depricated open_zarr

    Miguel Jimenez-Urias committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    bae7f10 View commit details
    Browse the repository at this point in the history

  18. defined engine=zarr

    Miguel Jimenez-Urias committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    37ff214 View commit details
    Browse the repository at this point in the history

  19. correct argument for open_dataset

    Miguel Jimenez-Urias committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    b8b98f5 View commit details
    Browse the repository at this point in the history

  20. pass arguments as backend_kwargs

    Miguel Jimenez-Urias committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    5c37329 View commit details
    Browse the repository at this point in the history

  21. pass backend_kwargs as argument

    Miguel Jimenez-Urias committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    831f15b View commit details
    Browse the repository at this point in the history

  22. typo

    Miguel Jimenez-Urias committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    80dd7da View commit details
    Browse the repository at this point in the history

  23. set `overwrite_enconded_chunks as backend_kwargs

    Miguel Jimenez-Urias committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    4ebf380 View commit details
    Browse the repository at this point in the history

  24. do not pass as backend, use for chunking

    Miguel Jimenez-Urias committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    4ce3007 View commit details
    Browse the repository at this point in the history

Commits on May 22, 2020

  1. removed commented code

    Miguel Jimenez-Urias committed May 22, 2020
    Configuration menu
    Copy the full SHA
    89a780b View commit details
    Browse the repository at this point in the history

  2. moved definitions to zarr backends

    Miguel Jimenez-Urias committed May 22, 2020
    Configuration menu
    Copy the full SHA
    6f6eb23 View commit details
    Browse the repository at this point in the history

  3. Merge pull request #1 from Mikejmnez/new_branch 

    moved definitions to zarr backends
    Miguel Jimenez authored May 22, 2020
    Configuration menu
    Copy the full SHA
    62893ab View commit details
    Browse the repository at this point in the history