Skip to content

Commit

Permalink
remove non-existant link, add sphinx-mermaid
Browse files Browse the repository at this point in the history
  • Loading branch information
scottyhq committed Jul 8, 2024
1 parent 8bbc512 commit 8eccb5d
Show file tree
Hide file tree
Showing 5 changed files with 83 additions and 24 deletions.
1 change: 1 addition & 0 deletions _config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ sphinx:
# https://github.com/wpilibsuite/sphinxext-rediraffe
- sphinxext.rediraffe
- sphinx_codeautolink
- sphinxcontrib.mermaid

config:
language: en # accessibility
Expand Down
96 changes: 76 additions & 20 deletions conda/conda-lock.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,10 @@
version: 1
metadata:
content_hash:
osx-64: 1791e916d80ac885466e643ee654367d8fb91d5a74d34ec6743c8b39aa52399e
linux-64: b96579b41e892b6d1a453495bcefb85681e0d807fe3e93866465367858319bcd
win-64: 9aba715d7beceef1f07fd4120597cdc813c0c3f6dd61e8562ad73da4136ad0b6
osx-arm64: fb501e34465ed4fac70f9444ad2d1f9a85d629db962c2e3cd5f58b68fb9fa538
osx-64: 49571e2dd9e6d47ce6aa3d5bbdc919081985983797be17081f68f976779c036e
linux-64: 4daa19bce4836dfdde9cc57bf64660e6a0a4f9cd3aec6d3431e4bed60e747064
win-64: 6748589622af59ba72a8bf19ac474488b96e495deb23fbd7fe5599db2c6fbf0b
osx-arm64: c99d388694ed3922bfb2c9f3f7467188f2730fb6a376021cd10f2c3468b709c5
channels:
- url: conda-forge
used_env_vars: []
Expand Down Expand Up @@ -8583,55 +8583,55 @@ package:
category: main
optional: false
- name: identify
version: 2.5.36
version: 2.6.0
manager: conda
platform: linux-64
dependencies:
python: '>=3.6'
ukkonen: ''
url: https://conda.anaconda.org/conda-forge/noarch/identify-2.5.36-pyhd8ed1ab_0.conda
url: https://conda.anaconda.org/conda-forge/noarch/identify-2.6.0-pyhd8ed1ab_0.conda
hash:
md5: ba68cb5105760379432cebc82b45af40
sha256: dc98ab2233d3ed3692499e2a06b027489ee317658cef9277ec23cab00236f31c
md5: f80cc5989f445f23b1622d6c455896d9
sha256: 4a2889027df94d51be283536ac235feba77eaa42a0d051f65cd07ba824b324a6
category: main
optional: false
- name: identify
version: 2.5.36
version: 2.6.0
manager: conda
platform: osx-64
dependencies:
ukkonen: ''
python: '>=3.6'
url: https://conda.anaconda.org/conda-forge/noarch/identify-2.5.36-pyhd8ed1ab_0.conda
url: https://conda.anaconda.org/conda-forge/noarch/identify-2.6.0-pyhd8ed1ab_0.conda
hash:
md5: ba68cb5105760379432cebc82b45af40
sha256: dc98ab2233d3ed3692499e2a06b027489ee317658cef9277ec23cab00236f31c
md5: f80cc5989f445f23b1622d6c455896d9
sha256: 4a2889027df94d51be283536ac235feba77eaa42a0d051f65cd07ba824b324a6
category: main
optional: false
- name: identify
version: 2.5.36
version: 2.6.0
manager: conda
platform: osx-arm64
dependencies:
ukkonen: ''
python: '>=3.6'
url: https://conda.anaconda.org/conda-forge/noarch/identify-2.5.36-pyhd8ed1ab_0.conda
url: https://conda.anaconda.org/conda-forge/noarch/identify-2.6.0-pyhd8ed1ab_0.conda
hash:
md5: ba68cb5105760379432cebc82b45af40
sha256: dc98ab2233d3ed3692499e2a06b027489ee317658cef9277ec23cab00236f31c
md5: f80cc5989f445f23b1622d6c455896d9
sha256: 4a2889027df94d51be283536ac235feba77eaa42a0d051f65cd07ba824b324a6
category: main
optional: false
- name: identify
version: 2.5.36
version: 2.6.0
manager: conda
platform: win-64
dependencies:
ukkonen: ''
python: '>=3.6'
url: https://conda.anaconda.org/conda-forge/noarch/identify-2.5.36-pyhd8ed1ab_0.conda
url: https://conda.anaconda.org/conda-forge/noarch/identify-2.6.0-pyhd8ed1ab_0.conda
hash:
md5: ba68cb5105760379432cebc82b45af40
sha256: dc98ab2233d3ed3692499e2a06b027489ee317658cef9277ec23cab00236f31c
md5: f80cc5989f445f23b1622d6c455896d9
sha256: 4a2889027df94d51be283536ac235feba77eaa42a0d051f65cd07ba824b324a6
category: main
optional: false
- name: idna
Expand Down Expand Up @@ -23683,6 +23683,62 @@ package:
sha256: d4337d83b8edba688547766fc80f1ac86d6ec86ceeeda93f376acc04079c5ce2
category: main
optional: false
- name: sphinxcontrib-mermaid
version: 0.9.2
manager: conda
platform: linux-64
dependencies:
docutils: ''
python: '>=3.7'
sphinx: ''
url: https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-mermaid-0.9.2-pyhd8ed1ab_0.conda
hash:
md5: 54a6a75e5b3989f1d925d8e5674bbbcb
sha256: bb02467bb3569406d978112f299e8d8b0832cc495b8bbd5d591858ddbe3a291d
category: main
optional: false
- name: sphinxcontrib-mermaid
version: 0.9.2
manager: conda
platform: osx-64
dependencies:
sphinx: ''
docutils: ''
python: '>=3.7'
url: https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-mermaid-0.9.2-pyhd8ed1ab_0.conda
hash:
md5: 54a6a75e5b3989f1d925d8e5674bbbcb
sha256: bb02467bb3569406d978112f299e8d8b0832cc495b8bbd5d591858ddbe3a291d
category: main
optional: false
- name: sphinxcontrib-mermaid
version: 0.9.2
manager: conda
platform: osx-arm64
dependencies:
sphinx: ''
docutils: ''
python: '>=3.7'
url: https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-mermaid-0.9.2-pyhd8ed1ab_0.conda
hash:
md5: 54a6a75e5b3989f1d925d8e5674bbbcb
sha256: bb02467bb3569406d978112f299e8d8b0832cc495b8bbd5d591858ddbe3a291d
category: main
optional: false
- name: sphinxcontrib-mermaid
version: 0.9.2
manager: conda
platform: win-64
dependencies:
sphinx: ''
docutils: ''
python: '>=3.7'
url: https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-mermaid-0.9.2-pyhd8ed1ab_0.conda
hash:
md5: 54a6a75e5b3989f1d925d8e5674bbbcb
sha256: bb02467bb3569406d978112f299e8d8b0832cc495b8bbd5d591858ddbe3a291d
category: main
optional: false
- name: sphinxcontrib-qthelp
version: 1.0.7
manager: conda
Expand Down
1 change: 1 addition & 0 deletions conda/environment-unpinned.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ dependencies:
- rioxarray
- scipy
- sphinx-codeautolink
- sphinxcontrib-mermaid
- sphinx-notfound-page
- sphinxext-rediraffe
- s3fs
Expand Down
5 changes: 3 additions & 2 deletions conda/environment.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Generated by conda-lock.
# platform: linux-64
# input_hash: b96579b41e892b6d1a453495bcefb85681e0d807fe3e93866465367858319bcd
# input_hash: 4daa19bce4836dfdde9cc57bf64660e6a0a4f9cd3aec6d3431e4bed60e747064

channels:
- conda-forge
Expand Down Expand Up @@ -151,7 +151,7 @@ dependencies:
- hvplot=0.10.0=pyhd8ed1ab_0
- hyperframe=6.0.1=pyhd8ed1ab_0
- icu=73.2=h59595ed_0
- identify=2.5.36=pyhd8ed1ab_0
- identify=2.6.0=pyhd8ed1ab_0
- idna=3.7=pyhd8ed1ab_0
- imagesize=1.4.1=pyhd8ed1ab_0
- importlib-metadata=8.0.0=pyha770c72_0
Expand Down Expand Up @@ -401,6 +401,7 @@ dependencies:
- sphinxcontrib-devhelp=1.0.6=pyhd8ed1ab_0
- sphinxcontrib-htmlhelp=2.0.5=pyhd8ed1ab_0
- sphinxcontrib-jsmath=1.0.1=pyhd8ed1ab_0
- sphinxcontrib-mermaid=0.9.2=pyhd8ed1ab_0
- sphinxcontrib-qthelp=1.0.7=pyhd8ed1ab_0
- sphinxcontrib-serializinghtml=1.1.10=pyhd8ed1ab_0
- sphinxext-rediraffe=0.2.7=pyhd8ed1ab_1
Expand Down
4 changes: 2 additions & 2 deletions intermediate/remote_data/remote-data.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,7 @@
"\n",
"**Handles Own I/O**: Some libraries have self contained code that can handle I/O, compression, codecs and data access. Some engines task their I/O to lower level libraries. This is the case with rasterio that uses GDAL to access raster files. If a Library is in control of its own I/O operations can be easily adapted to read from buffers.\n",
"\n",
"```mermaid\n",
"```{mermaid}\n",
"graph TD\n",
" A[\"netCDF-4 (.nc, .nc4) and most HDF5 files\"] -->|netcdf4| B[\"Remote Access: No\"]\n",
" A -->|h5netcdf| C[\"Remote Access: Yes\"]\n",
Expand All @@ -230,7 +230,7 @@
"\n",
"When we use fsspec to abstract a remote file we are in essence translating byte requests to HTTP range requests over the internet. An HTTP request is a costly I/O operation compared to accessing a local file. Because of this, it's common that libraries that handle over the network data transfers implement a cache to avoid requesting the same data over and over. In the case of fsspec there are different ways to ask the library to handle this **caching and this is one of the most relevant performance considerations** when we work with xarray and remote data.\n",
"\n",
"fsspec default cache is called `read-ahead` and as its name suggests it will read ahead of our request a fixed amount of bytes, this is good when we are working with text or tabular data but it's really an anti pattern when we work with scientific data formats. [Benchmarks show]() that any of the caching schemas will perform better than using the default `read-ahead`.\n",
"fsspec default cache is called `read-ahead` and as its name suggests it will read ahead of our request a fixed amount of bytes, this is good when we are working with text or tabular data but it's really an anti pattern when we work with scientific data formats. Benchmarks show that any of the caching schemas will perform better than using the default `read-ahead`.\n",
"\n",
"### fsspec caching implementations.\n",
"\n",
Expand Down

0 comments on commit 8eccb5d

Please sign in to comment.