zarr reads on the client don't work on latest cluster -- killed worker #265
Comments
|
Note that there are two different docker images, one for the notebook and
one for the workers. You probably need to build the worker image, push it,
and then list it in this file pangeo/gce/notebook/worker-template.yaml
Fortunately you can test this without upgrading the helm chart just by
modifying your own worker-template.yaml file (it should be in your home
directory). Eventually though this should be added to the image and then
pushed so that people get the right worker image by default.
…On Fri, May 18, 2018 at 4:58 PM, Ryan Abernathey ***@***.***> wrote:
When I go to actually load data from any of the zarr gcs datasets, I get
an error like this
---------------------------------------------------------------------------
KilledWorker Traceback (most recent call last)
<ipython-input-6-0247f79e60da> in <module>()
1 plt.rcParams['figure.figsize'] = (15, 8)
----> 2 ds.sla.sel(time='1982-08-07', method='nearest').plot()
/opt/conda/lib/python3.6/site-packages/xarray/plot/plot.py in __call__(self, **kwargs)
372
373 def __call__(self, **kwargs):
--> 374 return plot(self._da, **kwargs)
375
376 @functools.wraps(hist)
/opt/conda/lib/python3.6/site-packages/xarray/plot/plot.py in plot(darray, row, col, col_wrap, ax, rtol, subplot_kws, **kwargs)
155 kwargs['ax'] = ax
156
--> 157 return plotfunc(darray, **kwargs)
158
159
/opt/conda/lib/python3.6/site-packages/xarray/plot/plot.py in newplotfunc(darray, x, y, figsize, size, aspect, ax, row, col, col_wrap, xincrease, yincrease, add_colorbar, add_labels, vmin, vmax, cmap, center, robust, extend, levels, infer_intervals, colors, subplot_kws, cbar_ax, cbar_kwargs, **kwargs)
607
608 # Pass the data as a masked ndarray too
--> 609 zval = darray.to_masked_array(copy=False)
610
611 _ensure_plottable(xval, yval)
/opt/conda/lib/python3.6/site-packages/xarray/core/dataarray.py in to_masked_array(self, copy)
1453 Masked where invalid values (nan or inf) occur.
1454 """
-> 1455 isnull = pd.isnull(self.values)
1456 return np.ma.MaskedArray(data=self.values, mask=isnull, copy=copy)
1457
/opt/conda/lib/python3.6/site-packages/xarray/core/dataarray.py in values(self)
402 def values(self):
403 """The array's data as a numpy.ndarray"""
--> 404 return self.variable.values
405
406 @values.setter
/opt/conda/lib/python3.6/site-packages/xarray/core/variable.py in values(self)
385 def values(self):
386 """The variable's data as a numpy.ndarray"""
--> 387 return _as_array_or_item(self._data)
388
389 @values.setter
/opt/conda/lib/python3.6/site-packages/xarray/core/variable.py in _as_array_or_item(data)
209 TODO: remove this (replace with np.asarray) once these issues are fixed
210 """
--> 211 data = np.asarray(data)
212 if data.ndim == 0:
213 if data.dtype.kind == 'M':
/opt/conda/lib/python3.6/site-packages/numpy/core/numeric.py in asarray(a, dtype, order)
490
491 """
--> 492 return array(a, dtype, copy=False, order=order)
493
494
/opt/conda/lib/python3.6/site-packages/dask/array/core.py in __array__(self, dtype, **kwargs)
1190
1191 def __array__(self, dtype=None, **kwargs):
-> 1192 x = self.compute()
1193 if dtype and x.dtype != dtype:
1194 x = x.astype(dtype)
/opt/conda/lib/python3.6/site-packages/dask/base.py in compute(self, **kwargs)
152 dask.base.compute
153 """
--> 154 (result,) = compute(self, traverse=False, **kwargs)
155 return result
156
/opt/conda/lib/python3.6/site-packages/dask/base.py in compute(*args, **kwargs)
405 keys = [x.__dask_keys__() for x in collections]
406 postcomputes = [x.__dask_postcompute__() for x in collections]
--> 407 results = get(dsk, keys, **kwargs)
408 return repack([f(r, *a) for r, (f, a) in zip(results, postcomputes)])
409
/opt/conda/lib/python3.6/site-packages/distributed/client.py in get(self, dsk, keys, restrictions, loose_restrictions, resources, sync, asynchronous, direct, retries, priority, fifo_timeout, **kwargs)
2096 try:
2097 results = self.gather(packed, asynchronous=asynchronous,
-> 2098 direct=direct)
2099 finally:
2100 for f in futures.values():
/opt/conda/lib/python3.6/site-packages/distributed/client.py in gather(self, futures, errors, maxsize, direct, asynchronous)
1506 return self.sync(self._gather, futures, errors=errors,
1507 direct=direct, local_worker=local_worker,
-> 1508 asynchronous=asynchronous)
1509
1510 @gen.coroutine
/opt/conda/lib/python3.6/site-packages/distributed/client.py in sync(self, func, *args, **kwargs)
613 return future
614 else:
--> 615 return sync(self.loop, func, *args, **kwargs)
616
617 def __repr__(self):
/opt/conda/lib/python3.6/site-packages/distributed/utils.py in sync(loop, func, *args, **kwargs)
251 e.wait(10)
252 if error[0]:
--> 253 six.reraise(*error[0])
254 else:
255 return result[0]
/opt/conda/lib/python3.6/site-packages/six.py in reraise(tp, value, tb)
691 if value.__traceback__ is not tb:
692 raise value.with_traceback(tb)
--> 693 raise value
694 finally:
695 value = None
/opt/conda/lib/python3.6/site-packages/distributed/utils.py in f()
236 yield gen.moment
237 thread_state.asynchronous = True
--> 238 result[0] = yield make_coro()
239 except Exception as exc:
240 error[0] = sys.exc_info()
/opt/conda/lib/python3.6/site-packages/tornado/gen.py in run(self)
1097
1098 try:
-> 1099 value = future.result()
1100 except Exception:
1101 self.had_exception = True
/opt/conda/lib/python3.6/site-packages/tornado/gen.py in run(self)
1105 if exc_info is not None:
1106 try:
-> 1107 yielded = self.gen.throw(*exc_info)
1108 finally:
1109 # Break up a reference to itself
/opt/conda/lib/python3.6/site-packages/distributed/client.py in _gather(self, futures, errors, direct, local_worker)
1383 six.reraise(type(exception),
1384 exception,
-> 1385 traceback)
1386 if errors == 'skip':
1387 bad_keys.add(key)
/opt/conda/lib/python3.6/site-packages/six.py in reraise(tp, value, tb)
691 if value.__traceback__ is not tb:
692 raise value.with_traceback(tb)
--> 693 raise value
694 finally:
695 value = None
KilledWorker: ('zarr-30761e7bf266279b7254c29da310d0bd', 'tcp://10.20.144.7:43398')
😭
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#265>, or mute the thread
<https://github.com/notifications/unsubscribe-auth/AASszOq2bjvO9NE-roreheUekQ23ZBqJks5tzzYSgaJpZM4UFWQF>
.
|
|
In #261 I did update and rebuild the worker image and update the corresponding tag in the notebook image.
…Sent from my iPhone
On May 18, 2018, at 5:20 PM, Matthew Rocklin ***@***.***> wrote:
Note that there are two different docker images, one for the notebook and
one for the workers. You probably need to build the worker image, push it,
and then list it in this file pangeo/gce/notebook/worker-template.yaml
Fortunately you can test this without upgrading the helm chart just by
modifying your own worker-template.yaml file (it should be in your home
directory). Eventually though this should be added to the image and then
pushed so that people get the right worker image by default.
On Fri, May 18, 2018 at 4:58 PM, Ryan Abernathey ***@***.***>
wrote:
> When I go to actually load data from any of the zarr gcs datasets, I get
> an error like this
>
> ---------------------------------------------------------------------------
> KilledWorker Traceback (most recent call last)
> <ipython-input-6-0247f79e60da> in <module>()
> 1 plt.rcParams['figure.figsize'] = (15, 8)
> ----> 2 ds.sla.sel(time='1982-08-07', method='nearest').plot()
>
> /opt/conda/lib/python3.6/site-packages/xarray/plot/plot.py in __call__(self, **kwargs)
> 372
> 373 def __call__(self, **kwargs):
> --> 374 return plot(self._da, **kwargs)
> 375
> 376 @functools.wraps(hist)
>
> /opt/conda/lib/python3.6/site-packages/xarray/plot/plot.py in plot(darray, row, col, col_wrap, ax, rtol, subplot_kws, **kwargs)
> 155 kwargs['ax'] = ax
> 156
> --> 157 return plotfunc(darray, **kwargs)
> 158
> 159
>
> /opt/conda/lib/python3.6/site-packages/xarray/plot/plot.py in newplotfunc(darray, x, y, figsize, size, aspect, ax, row, col, col_wrap, xincrease, yincrease, add_colorbar, add_labels, vmin, vmax, cmap, center, robust, extend, levels, infer_intervals, colors, subplot_kws, cbar_ax, cbar_kwargs, **kwargs)
> 607
> 608 # Pass the data as a masked ndarray too
> --> 609 zval = darray.to_masked_array(copy=False)
> 610
> 611 _ensure_plottable(xval, yval)
>
> /opt/conda/lib/python3.6/site-packages/xarray/core/dataarray.py in to_masked_array(self, copy)
> 1453 Masked where invalid values (nan or inf) occur.
> 1454 """
> -> 1455 isnull = pd.isnull(self.values)
> 1456 return np.ma.MaskedArray(data=self.values, mask=isnull, copy=copy)
> 1457
>
> /opt/conda/lib/python3.6/site-packages/xarray/core/dataarray.py in values(self)
> 402 def values(self):
> 403 """The array's data as a numpy.ndarray"""
> --> 404 return self.variable.values
> 405
> 406 @values.setter
>
> /opt/conda/lib/python3.6/site-packages/xarray/core/variable.py in values(self)
> 385 def values(self):
> 386 """The variable's data as a numpy.ndarray"""
> --> 387 return _as_array_or_item(self._data)
> 388
> 389 @values.setter
>
> /opt/conda/lib/python3.6/site-packages/xarray/core/variable.py in _as_array_or_item(data)
> 209 TODO: remove this (replace with np.asarray) once these issues are fixed
> 210 """
> --> 211 data = np.asarray(data)
> 212 if data.ndim == 0:
> 213 if data.dtype.kind == 'M':
>
> /opt/conda/lib/python3.6/site-packages/numpy/core/numeric.py in asarray(a, dtype, order)
> 490
> 491 """
> --> 492 return array(a, dtype, copy=False, order=order)
> 493
> 494
>
> /opt/conda/lib/python3.6/site-packages/dask/array/core.py in __array__(self, dtype, **kwargs)
> 1190
> 1191 def __array__(self, dtype=None, **kwargs):
> -> 1192 x = self.compute()
> 1193 if dtype and x.dtype != dtype:
> 1194 x = x.astype(dtype)
>
> /opt/conda/lib/python3.6/site-packages/dask/base.py in compute(self, **kwargs)
> 152 dask.base.compute
> 153 """
> --> 154 (result,) = compute(self, traverse=False, **kwargs)
> 155 return result
> 156
>
> /opt/conda/lib/python3.6/site-packages/dask/base.py in compute(*args, **kwargs)
> 405 keys = [x.__dask_keys__() for x in collections]
> 406 postcomputes = [x.__dask_postcompute__() for x in collections]
> --> 407 results = get(dsk, keys, **kwargs)
> 408 return repack([f(r, *a) for r, (f, a) in zip(results, postcomputes)])
> 409
>
> /opt/conda/lib/python3.6/site-packages/distributed/client.py in get(self, dsk, keys, restrictions, loose_restrictions, resources, sync, asynchronous, direct, retries, priority, fifo_timeout, **kwargs)
> 2096 try:
> 2097 results = self.gather(packed, asynchronous=asynchronous,
> -> 2098 direct=direct)
> 2099 finally:
> 2100 for f in futures.values():
>
> /opt/conda/lib/python3.6/site-packages/distributed/client.py in gather(self, futures, errors, maxsize, direct, asynchronous)
> 1506 return self.sync(self._gather, futures, errors=errors,
> 1507 direct=direct, local_worker=local_worker,
> -> 1508 asynchronous=asynchronous)
> 1509
> 1510 @gen.coroutine
>
> /opt/conda/lib/python3.6/site-packages/distributed/client.py in sync(self, func, *args, **kwargs)
> 613 return future
> 614 else:
> --> 615 return sync(self.loop, func, *args, **kwargs)
> 616
> 617 def __repr__(self):
>
> /opt/conda/lib/python3.6/site-packages/distributed/utils.py in sync(loop, func, *args, **kwargs)
> 251 e.wait(10)
> 252 if error[0]:
> --> 253 six.reraise(*error[0])
> 254 else:
> 255 return result[0]
>
> /opt/conda/lib/python3.6/site-packages/six.py in reraise(tp, value, tb)
> 691 if value.__traceback__ is not tb:
> 692 raise value.with_traceback(tb)
> --> 693 raise value
> 694 finally:
> 695 value = None
>
> /opt/conda/lib/python3.6/site-packages/distributed/utils.py in f()
> 236 yield gen.moment
> 237 thread_state.asynchronous = True
> --> 238 result[0] = yield make_coro()
> 239 except Exception as exc:
> 240 error[0] = sys.exc_info()
>
> /opt/conda/lib/python3.6/site-packages/tornado/gen.py in run(self)
> 1097
> 1098 try:
> -> 1099 value = future.result()
> 1100 except Exception:
> 1101 self.had_exception = True
>
> /opt/conda/lib/python3.6/site-packages/tornado/gen.py in run(self)
> 1105 if exc_info is not None:
> 1106 try:
> -> 1107 yielded = self.gen.throw(*exc_info)
> 1108 finally:
> 1109 # Break up a reference to itself
>
> /opt/conda/lib/python3.6/site-packages/distributed/client.py in _gather(self, futures, errors, direct, local_worker)
> 1383 six.reraise(type(exception),
> 1384 exception,
> -> 1385 traceback)
> 1386 if errors == 'skip':
> 1387 bad_keys.add(key)
>
> /opt/conda/lib/python3.6/site-packages/six.py in reraise(tp, value, tb)
> 691 if value.__traceback__ is not tb:
> 692 raise value.with_traceback(tb)
> --> 693 raise value
> 694 finally:
> 695 value = None
>
> KilledWorker: ('zarr-30761e7bf266279b7254c29da310d0bd', 'tcp://10.20.144.7:43398')
>
> 😭
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <#265>, or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AASszOq2bjvO9NE-roreheUekQ23ZBqJks5tzzYSgaJpZM4UFWQF>
> .
>
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.
|
|
Ah, sorry. I hadn't pulled
On Fri, May 18, 2018 at 5:28 PM, Ryan Abernathey <[email protected]>
wrote:
… In #261 I did update and rebuild the worker image and update the
corresponding tag in the notebook image.
Sent from my iPhone
> On May 18, 2018, at 5:20 PM, Matthew Rocklin ***@***.***>
wrote:
>
> Note that there are two different docker images, one for the notebook and
> one for the workers. You probably need to build the worker image, push
it,
> and then list it in this file pangeo/gce/notebook/worker-template.yaml
>
> Fortunately you can test this without upgrading the helm chart just by
> modifying your own worker-template.yaml file (it should be in your home
> directory). Eventually though this should be added to the image and then
> pushed so that people get the right worker image by default.
>
> On Fri, May 18, 2018 at 4:58 PM, Ryan Abernathey <
***@***.***>
> wrote:
>
> > When I go to actually load data from any of the zarr gcs datasets, I
get
> > an error like this
> >
> > ------------------------------------------------------------
---------------
> > KilledWorker Traceback (most recent call last)
> > <ipython-input-6-0247f79e60da> in <module>()
> > 1 plt.rcParams['figure.figsize'] = (15, 8)
> > ----> 2 ds.sla.sel(time='1982-08-07', method='nearest').plot()
> >
> > /opt/conda/lib/python3.6/site-packages/xarray/plot/plot.py in
__call__(self, **kwargs)
> > 372
> > 373 def __call__(self, **kwargs):
> > --> 374 return plot(self._da, **kwargs)
> > 375
> > 376 @functools.wraps(hist)
> >
> > /opt/conda/lib/python3.6/site-packages/xarray/plot/plot.py in
plot(darray, row, col, col_wrap, ax, rtol, subplot_kws, **kwargs)
> > 155 kwargs['ax'] = ax
> > 156
> > --> 157 return plotfunc(darray, **kwargs)
> > 158
> > 159
> >
> > /opt/conda/lib/python3.6/site-packages/xarray/plot/plot.py in
newplotfunc(darray, x, y, figsize, size, aspect, ax, row, col, col_wrap,
xincrease, yincrease, add_colorbar, add_labels, vmin, vmax, cmap, center,
robust, extend, levels, infer_intervals, colors, subplot_kws, cbar_ax,
cbar_kwargs, **kwargs)
> > 607
> > 608 # Pass the data as a masked ndarray too
> > --> 609 zval = darray.to_masked_array(copy=False)
> > 610
> > 611 _ensure_plottable(xval, yval)
> >
> > /opt/conda/lib/python3.6/site-packages/xarray/core/dataarray.py in
to_masked_array(self, copy)
> > 1453 Masked where invalid values (nan or inf) occur.
> > 1454 """
> > -> 1455 isnull = pd.isnull(self.values)
> > 1456 return np.ma.MaskedArray(data=self.values, mask=isnull,
copy=copy)
> > 1457
> >
> > /opt/conda/lib/python3.6/site-packages/xarray/core/dataarray.py in
values(self)
> > 402 def values(self):
> > 403 """The array's data as a numpy.ndarray"""
> > --> 404 return self.variable.values
> > 405
> > 406 @values.setter
> >
> > /opt/conda/lib/python3.6/site-packages/xarray/core/variable.py in
values(self)
> > 385 def values(self):
> > 386 """The variable's data as a numpy.ndarray"""
> > --> 387 return _as_array_or_item(self._data)
> > 388
> > 389 @values.setter
> >
> > /opt/conda/lib/python3.6/site-packages/xarray/core/variable.py in
_as_array_or_item(data)
> > 209 TODO: remove this (replace with np.asarray) once these issues are
fixed
> > 210 """
> > --> 211 data = np.asarray(data)
> > 212 if data.ndim == 0:
> > 213 if data.dtype.kind == 'M':
> >
> > /opt/conda/lib/python3.6/site-packages/numpy/core/numeric.py in
asarray(a, dtype, order)
> > 490
> > 491 """
> > --> 492 return array(a, dtype, copy=False, order=order)
> > 493
> > 494
> >
> > /opt/conda/lib/python3.6/site-packages/dask/array/core.py in
__array__(self, dtype, **kwargs)
> > 1190
> > 1191 def __array__(self, dtype=None, **kwargs):
> > -> 1192 x = self.compute()
> > 1193 if dtype and x.dtype != dtype:
> > 1194 x = x.astype(dtype)
> >
> > /opt/conda/lib/python3.6/site-packages/dask/base.py in compute(self,
**kwargs)
> > 152 dask.base.compute
> > 153 """
> > --> 154 (result,) = compute(self, traverse=False, **kwargs)
> > 155 return result
> > 156
> >
> > /opt/conda/lib/python3.6/site-packages/dask/base.py in compute(*args,
**kwargs)
> > 405 keys = [x.__dask_keys__() for x in collections]
> > 406 postcomputes = [x.__dask_postcompute__() for x in collections]
> > --> 407 results = get(dsk, keys, **kwargs)
> > 408 return repack([f(r, *a) for r, (f, a) in zip(results,
postcomputes)])
> > 409
> >
> > /opt/conda/lib/python3.6/site-packages/distributed/client.py in
get(self, dsk, keys, restrictions, loose_restrictions, resources, sync,
asynchronous, direct, retries, priority, fifo_timeout, **kwargs)
> > 2096 try:
> > 2097 results = self.gather(packed, asynchronous=asynchronous,
> > -> 2098 direct=direct)
> > 2099 finally:
> > 2100 for f in futures.values():
> >
> > /opt/conda/lib/python3.6/site-packages/distributed/client.py in
gather(self, futures, errors, maxsize, direct, asynchronous)
> > 1506 return self.sync(self._gather, futures, errors=errors,
> > 1507 direct=direct, local_worker=local_worker,
> > -> 1508 asynchronous=asynchronous)
> > 1509
> > 1510 @gen.coroutine
> >
> > /opt/conda/lib/python3.6/site-packages/distributed/client.py in
sync(self, func, *args, **kwargs)
> > 613 return future
> > 614 else:
> > --> 615 return sync(self.loop, func, *args, **kwargs)
> > 616
> > 617 def __repr__(self):
> >
> > /opt/conda/lib/python3.6/site-packages/distributed/utils.py in
sync(loop, func, *args, **kwargs)
> > 251 e.wait(10)
> > 252 if error[0]:
> > --> 253 six.reraise(*error[0])
> > 254 else:
> > 255 return result[0]
> >
> > /opt/conda/lib/python3.6/site-packages/six.py in reraise(tp, value,
tb)
> > 691 if value.__traceback__ is not tb:
> > 692 raise value.with_traceback(tb)
> > --> 693 raise value
> > 694 finally:
> > 695 value = None
> >
> > /opt/conda/lib/python3.6/site-packages/distributed/utils.py in f()
> > 236 yield gen.moment
> > 237 thread_state.asynchronous = True
> > --> 238 result[0] = yield make_coro()
> > 239 except Exception as exc:
> > 240 error[0] = sys.exc_info()
> >
> > /opt/conda/lib/python3.6/site-packages/tornado/gen.py in run(self)
> > 1097
> > 1098 try:
> > -> 1099 value = future.result()
> > 1100 except Exception:
> > 1101 self.had_exception = True
> >
> > /opt/conda/lib/python3.6/site-packages/tornado/gen.py in run(self)
> > 1105 if exc_info is not None:
> > 1106 try:
> > -> 1107 yielded = self.gen.throw(*exc_info)
> > 1108 finally:
> > 1109 # Break up a reference to itself
> >
> > /opt/conda/lib/python3.6/site-packages/distributed/client.py in
_gather(self, futures, errors, direct, local_worker)
> > 1383 six.reraise(type(exception),
> > 1384 exception,
> > -> 1385 traceback)
> > 1386 if errors == 'skip':
> > 1387 bad_keys.add(key)
> >
> > /opt/conda/lib/python3.6/site-packages/six.py in reraise(tp, value,
tb)
> > 691 if value.__traceback__ is not tb:
> > 692 raise value.with_traceback(tb)
> > --> 693 raise value
> > 694 finally:
> > 695 value = None
> >
> > KilledWorker: ('zarr-30761e7bf266279b7254c29da310d0bd', 'tcp://
10.20.144.7:43398')
> >
> > 😭
> >
> > —
> > You are receiving this because you are subscribed to this thread.
> > Reply to this email directly, view it on GitHub
> > <#265>, or mute the thread
> > <https://github.com/notifications/unsubscribe-auth/AASszOq2bjvO9NE-
roreheUekQ23ZBqJks5tzzYSgaJpZM4UFWQF>
> > .
> >
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub, or mute the thread.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#265 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AASszIVRSwmGT_vY5s05Hdg9dLYJFOk3ks5tzzzugaJpZM4UFWQF>
.
|
|
error from the worker |
|
Ok, false alarm. I discovered that there are different versions of gcsfs on notebook and worker. I'm guessing this was the cause of the problem. The worker has Unless anyone else has a similar problem, I'm pretty sure this was unique to me. |
|
I'm having a very similar issue on ocean.pangeo.io. I'm trying to use It takes ages for the scheduler to map a task on 20 blocks. And then the cluster seems to died with no feedback on the diagnostics page. In the notebook I see I found on another issue here and that showed me the following log: @rabernat Any ideas on what to do here? |
|
@dcherian can you do a basic operation like |
|
@dcherian - these issues are hard to debug. If you want help, it's best to post a fully reproducible example. We all have access to the same data and environment on ocean.pangeo.io, so it's relatively easy for someone else to look into the problem. |
|
Thanks @rabernat & @TomAugspurger It turns out that my issue stemmed from trying to pass an xarray |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When I go to actually load data from any of the zarr gcs datasets, I get an error like this
😭
The text was updated successfully, but these errors were encountered: