Skip to content

Commit

Permalink
Add Hugging Face hf:// to the list of fsspec compatible remote se…
Browse files Browse the repository at this point in the history
…rvices (#11012)
  • Loading branch information
lhoestq authored Mar 25, 2024
1 parent 8c2199e commit 4688222
Showing 1 changed file with 3 additions and 0 deletions.
3 changes: 3 additions & 0 deletions docs/source/how-to/connect-to-remote-data.rst
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ codebase:

- **Microsoft Azure Storage**: ``adl://``, ``abfs://`` or ``az://`` - Microsoft Azure Storage using adlfs_.

- **Hugging Face**: ``hf://`` - Hugging Face Hub of datasets for AI, using the huggingface_hub_ library.

- **HTTP(s)**: ``http://`` or ``https://`` for reading data directly from HTTP web servers.

`fsspec`_ also provides other file systems that may be of interest to Dask users, such as
Expand All @@ -48,6 +50,7 @@ file system is assumed (same as ``file://``).
.. _adlfs: https://github.com/dask/adlfs
.. _gcsfs: https://gcsfs.readthedocs.io/en/latest/
.. _PyArrow: https://arrow.apache.org/docs/python/
.. _huggingface_hub: https://huggingface.co/docs/huggingface_hub

Lower-level details on how Dask handles remote data is described
below in the Internals section
Expand Down

0 comments on commit 4688222

Please sign in to comment.