add hidden cell to install colab missing deps

[ocefpaf]

For future reference I used this code to find the missing modules on GoogleColab and add the extra cell to the notebooks:

from pathlib import Path

from depfinder import notebook_path_to_dependencies

notebook_path = Path(".")
notebooks = list(notebook_path.glob("**/*.ipynb"))

## Get modules that are absent on GoogleColab
# required = []

# for notebook in notebooks:
#     try:
#         ret = notebook_path_to_dependencies(notebook)
#     except Exception as err:
#         print(f"Could not parse {notebook=}. {err}")
#     required.extend(ret["required"])

# modules = sorted(set(required))

## go to colab with this list and run to obtain the absent list

# """
# import importlib
# for module in modules:
#     if importlib.util.find_spec(module) is None:
#         print(module)
# """

## update notebooks
import nbformat

absent = [
    "bagit",
    "cartopy",
    "cf-units",
    "cf_xarray",
    "compliance-checker",
    "erddapy",
    "geoplot",
    "gridgeo",
    "ioos-tools",
    "ioos_qc",
    "ipyleaflet",
    "iris",
    "netcdf4",
    "oceans",
    "odvc",
    "owslib",
    "palettable",
    "pocean-core",
    "pyobis",
    "pyoos",
    "pysgrid",
    "pyugrid",
    "pyworms",
    "retrying",
    "seawater",
    "zarr",
]


code = """\
import subprocess
import sys
COLAB = "google.colab" in sys.modules

def _install(package):
    if COLAB:
        ans = input(f"Install {{ package }}? [y/n]:")
        if ans.lower() in ["y", "yes"]:
            subprocess.check_call([sys.executable, "-m", "pip", "install", "--quiet", package])
            print(f"{{ package }} installed!")

def _colab_install_missing_deps(deps):
    import importlib
    for dep in deps:
        if importlib.util.find_spec(dep) is None:
            if dep == "iris":
                dep = "scitools-iris"
            _install(dep)

deps = {}
_colab_install_missing_deps(deps)"""


def update_notebook(notebook, code):
    nb = nbformat.read(notebook, as_version=4)

    new_cell = nbformat.v4.new_code_cell(code)
    new_cell.pop("id")  # cannot save mod nb with this
    new_cell["metadata"] = {"tags": ["remove-cell"]}  # won't render on jupyterbook
    nb.cells.insert(0, new_cell)
    nbformat.write(nb, notebook, version=4)


for notebook in notebooks:
    try:
        required = notebook_path_to_dependencies(notebook)["required"]
        missing_from_colab = [module for module in absent if module in required]
        update_notebook(notebook, code.format(missing_from_colab))
    except Exception as err:
        print(f"Could not parse {notebook=}. {err}")
        continue

[ocefpaf]

The jupyterbook/content/code_gallery/data_analysis_and_visualization_notebooks/2021-10-25-ERDDAP-interpolate.ipynb is failing with a 403 b/c that ERDDAP server doesn't like to be accessed and prohibits folks who try to use it :-/

It is safe to merge but we should consider using a more forgiving server for the interpolation example, one that likes users and let them access data.

[ocefpaf]

@MathewBiddle this one is ready to go. It kinds of touches almost all the notebooks, adding the GoogleColab dependency check cell, so, to avoid conflicts, it would be nice to get this in before any other PR that edits the notebooks.

[MathewBiddle]

Good to know about that ERDDAP server. It might be worthwhile to see if we can use another one that isn't as restrictive.

As far as the changes to the notebooks go, I have a question. If I cloned the repo and tried to run one of the notebooks with this install cell on my local system, would this step install the library on my machine?

I'm thinking of the case where I activate the IOOS env (maybe I'm in a different env), would that cell then install those dependencies to my current working env?

[ocefpaf]

As far as the changes to the notebooks go, I have a question. If I cloned the repo and tried to run one of the notebooks with this install cell on my local system, would this step install the library on my machine?

It should not install anything that importlib can find:

        if importlib.util.find_spec(dep) is None:  # this is none only if importlib could not find it in the env
            _install(dep)

I'm thinking of the case where I activate the IOOS env (maybe I'm in a different env), would that cell then install those dependencies to my current working env?

In that case, yes, b/c the dependency is not there. We can add y/n question to prevent a silent installation without the user response.

[ocefpaf]

@MathewBiddle I added two safety measures there:

• only run that code if in GoogleCoLab
• always ask y/n before installing

I also fixed a bug where iris (module name but not package name on PyPI) was installed instead of scitools-iris.

The new cell we are adding is:

import subprocess
import sys

COLAB = "google.colab" in sys.modules


def _install(package):
    if COLAB:
        ans = input(f"Install { package }? [y/n]:")
        if ans.lower() in ["y", "yes"]:
            subprocess.check_call(
                [sys.executable, "-m", "pip", "install", "--quiet", package]
            )
            print(f"{ package } installed!")


def _colab_install_missing_deps(deps):
    import importlib

    for dep in deps:
        if importlib.util.find_spec(dep) is None:
            if dep == "iris":
                dep = "scitools-iris"
            _install(dep)

[MathewBiddle]

I like it!