Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

issues discovered by the all-but-dask CI #3921

Closed
3 tasks done
keewis opened this issue Mar 30, 2020 · 4 comments · Fixed by #8674
Closed
3 tasks done

issues discovered by the all-but-dask CI #3921

keewis opened this issue Mar 30, 2020 · 4 comments · Fixed by #8674
Labels

Comments

@keewis
Copy link
Collaborator

keewis commented Mar 30, 2020

After adding the py38-all-but-dask CI in #3919, it discovered a few backend issues:

@max-sixty
Copy link
Collaborator

Great finds @keewis !

Any idea why a lock is being serialized at all with a Dataset?

@keewis
Copy link
Collaborator Author

keewis commented Mar 31, 2020

I think the lock is used to synchronize the I/O on the rasterio file, so we can't really remove it. The failing test states it is a regression test for #2121, but since then the backend code was rewritten. We'd probably need a serializable lock that does not depend on dask (I don't know much about the backend code, though)

Edit: we could vendor dask's SerializableLock, which basically wraps a threading.Lock but also makes sure to use the same lock after every pickle dumps/loads cycle.

@headtr1ck
Copy link
Collaborator

I think the last remaining item has been fixed by #7586, but not completely sure.

Then we could close this issue :)

@keewis
Copy link
Collaborator Author

keewis commented Apr 6, 2023

I think we'd need to be able to remove this xfail first:

@pytest.mark.xfail(
reason="zarr without dask handles negative steps in slices incorrectly",
)
I didn't check, this could be xpassing right now.

dcherian added a commit to dcherian/xarray that referenced this issue Jan 26, 2024
dcherian added a commit to dcherian/xarray that referenced this issue Jan 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants