ZI drivers + Rarr implementation

[joshmoore]

In an attempt to:

• speed up tests
• make them more manageable
• ease local testing
• encourage the creation of more tests
• clearly identify what each implementation supports
• etc.

This PR:

• moves each implementation to its own subdirectory
• creates a conda environment for each
• runs each in a separate GHA job
• generates an output page for each
  • or alternatively: Python support dorny/test-reporter#244
• ... with timings

[joshmoore]

Various updates. make list now prints this:

../../data/js.zr	gzip
../../data/js.zr	blosc/lz4
../../data/js.zr	zlib
../../data/js.zr	raw
../../data/jzarr_flat.zr	raw
../../data/jzarr_flat.zr	zlib
../../data/jzarr_flat.zr	blosc/lz4
../../data/n5-java.n5	raw
../../data/n5-java.n5	bzip2
../../data/n5-java.n5	gzip
../../data/n5-java.n5	lz4
../../data/n5-java.n5	xz
../../data/pyn5.n5	raw
../../data/pyn5.n5	bzip2
../../data/pyn5.n5	gzip
../../data/z5py.zr	gzip
../../data/z5py.zr	blosc/lz4
../../data/z5py.zr	zlib
../../data/z5py.zr	raw
../../data/z5py.n5	gzip
../../data/z5py.n5	raw
../../data/zarr_DirectoryStore_flat.zr	gzip
../../data/zarr_DirectoryStore_flat.zr	blosc/lz4
../../data/zarr_DirectoryStore_flat.zr	zlib
../../data/zarr_DirectoryStore_flat.zr	raw
../../data/zarr_FSStore_flat.zr	gzip
../../data/zarr_FSStore_flat.zr	blosc/lz4
../../data/zarr_FSStore_flat.zr	zlib
../../data/zarr_FSStore_flat.zr	raw
../../data/zarr_NestedDirectoryStore_nested.zr	gzip
../../data/zarr_NestedDirectoryStore_nested.zr	blosc/lz4
../../data/zarr_NestedDirectoryStore_nested.zr	zlib
../../data/zarr_NestedDirectoryStore_nested.zr	raw
../../data/zarr_FSStore_nested.zr	gzip
../../data/zarr_FSStore_nested.zr	blosc/lz4
../../data/zarr_FSStore_nested.zr	zlib
../../data/zarr_FSStore_nested.zr	raw
../../data/zarr.n5	gzip
../../data/zarr.n5	raw
../../data/zarrita	gzip
../../data/zarrita	blosc/lz4
../../data/zarrita	raw
../../data/zarrita_sharded	gzip
../../data/zarrita_sharded	blosc/lz4
../../data/zarrita_sharded	raw
../../data/zarrita_nested	gzip
../../data/zarrita_nested	blosc/lz4
../../data/zarrita_nested	raw
../../data/zarrita_nested_sharded	gzip
../../data/zarrita_nested_sharded	blosc/lz4
../../data/zarrita_nested_sharded	raw

[joshmoore]

Locally I'm now seeing this:

$ pytest test/test_read_all.py
====================================================================================
platform darwin -- Python 3.12.4, pytest-8.3.1, pluggy-1.5.0
rootdir: /Users/jamoore/opt/zarr_implementations
collected 41 items

test/test_read_all.py ...ssss....ssssssss...sss...sssssssssss..                                                                                                                                                        [100%]

====================================================================================

I will likely need to add a few .skip files (e.g., for pyn5) but the current failures on GitHub actions are across the board:

import-im6.q16: unable to open X server `' @ error/import.c/ImportImageCommand/346.

Still trying to track that down if anyone has seen it before.

[joshmoore]

============================= test session starts ==============================
platform linux -- Python 3.12.4, pytest-8.3.1, pluggy-1.5.0
rootdir: /home/runner/work/zarr_implementations/zarr_implementations
collected 123 items

test/test_read_all.py ...ssss....ssssssss...sss...ssssss...ssss....sssss [ 40%]
sss...ssss....ssssssss...ssss....ssssssss...ssss....sssssssssssss..ssss. [ 99%]
.                                                                        [100%]

================== 45 passed, 78 skipped in 113.97s (0:01:53) ==================

I'd propose getting this merged and then start adding in other implementations including converters like https://github.com/glencoesoftware/zarr2zarr and https://github.com/LDeakin/zarrs_tools

cc: @MSanKeys963 @sbesson

[MSanKeys963]

I'd propose getting this merged and then start adding in other implementations including converters like https://github.com/glencoesoftware/zarr2zarr and https://github.com/LDeakin/zarrs_tools

Would having a page like https://zarr.dev/tools and listing the converters make sense?

[joshmoore]

Would having a page like https://zarr.dev/tools and listing the converters make sense?

Eventually. Let's get some more of the testing done first though.

[joshmoore]

cc: @perlman