Add tokens to the collections

[mgrover1]

It would be helpful, especially when cacheing datasets, to have a list of corresponding tokens for each dataset in a collection.

These could be auto-generated by dask.base.tokenize

[andersy005]

These could be auto-generated by dask.base.tokenize

I'm afraid dask.base.tokenize isn't deterministic for xarray objects:

In [2]: import dask, xarray as xr, xcollection as xc

In [3]: ds = xr.tutorial.open_dataset('rasm')

In [4]: dask.base.tokenize(ds) == dask.base.tokenize(ds)
Out[4]: False

In [5]: dask.base.tokenize(ds) == dask.base.tokenize(ds)
Out[5]: False

The issue appears to be caused by the coordinates:

In [8]: dask.base.tokenize(ds.Tair.data) == dask.base.tokenize(ds.Tair.data)
Out[8]: True

In [16]: dask.base.tokenize(ds.Tair._coords) == dask.base.tokenize(ds.Tair._coords)
Out[16]: False

[mgrover1]

Hmmm... I wonder if there would be any other options?

[andersy005]

Hmmm... I wonder if there would be any other options?

I just re-run the example above, and things seem to work just fine: I have no idea what's going on.. pydata/xarray#4738 (comment)

[mgrover1]

That is quite odd... glad it's running okay now?