Nbless: a Python package for programmatic Jupyter notebook workflows

Introduction

The nbless Python package allows you to (de)construct, convert, and execute Jupyter Notebooks in

• your terminal (e.g. bash, zsh, fish, etc.) or
• your favorite Python environment (e.g. PyCharm or Visual Studio Code).

The nbless Python package consists of 6 Python functions and shell commands:

• nbconv, which converts a notebook into various formats.
• nbdeck, which prepares a notebook to be viewed as or converted into slides.
• nbexec, which runs a notebook from top to bottom and saves an executed version.
• nbless, which calls nbuild and nbexec to create and execute a notebook.
• nbraze, which extracts code and markdown files from a notebook.
• nbuild, which creates a notebook from source files, e.g. Python (.py) and R (.R) scripts, markdown (.md), and text (.txt) files.

For a related package that provides programmatic tools for working with R Markdown (Rmd) files, check out the Rmdawn Python package.

Documentation and code

The documentation is hosted at https://py4ds.github.io/nbless/.

The code is hosted at https://github.com/py4ds/nbless.

Installation

pip install nbless

or clone the repo, e.g. git clone https://github.com/py4ds/nbless and install locally using setup.py (python setup.py install) or pip (pip install .).

Usage

Converting Jupyter notebooks with nbconv

The nbconv shell command can export a notebook to many different formats using the nbconvert library. Converting to all formats except HTML requires pandoc. Exporting to PDF requires LaTeX.

The supported exporters are

• asciidoc
• pdf
• html
• latex
• markdown
• python
• rst
• script
• slides

For example, nbconv can create a python script by extracting the content from code cells and discarding all output and markdown content.

nbconv notebook.ipynb --exporter python
# Or
nbconv notebook.ipynb -e python

In the example above, the notebook would be printed to the screen (stdout), but you can create or overwrite a notebook files with the --out_file or -o flag:

nbconv notebook.ipynb --out_file script.py
# Or
nbconv notebook.ipynb -o script.py

If the exporter is not provided, nbconv will try to infer the exporter type from the out_file extension.

If neither the exporter or out_file arguments are provided, the exporter will be set to html.

nbconv notebook.ipynb

Unlike the shell command, the nbconv Python function does not create a file on its own. To create a converted file with Python, use the pathlib library.

from pathlib import Path
from nbless import nbconv

# Create notebook.py from notebook.ipynb
filename, contents = nbconv("notebook.ipynb", "python")
Path(filename).write_text(contents)

# Create report.html from notebook.ipynb
filename, contents = nbconv("notebook.ipynb", "html")
Path('report.html').write_text(contents)

Creating HTML slides with nbdeck and nbconv

With nbdeck, you can prepare HTML slides from a Jupyter notebook.

nbdeck notebook.ipynb -o slides.ipynb
nbconv slides.ipynb  -e slides -o slides.html

You can run nbdeck without nbconv, if you do not want to create HTML slides and instead want to use nbviewer or the RISE extension. If an out_file name is not provided, the notebook file contents will be printed. You can provide a more descriptive name for the executed output notebook with the --out_file or -o flag or by redirecting the output to a file with >.

nbdeck notebook.ipynb --out_file slides.ipynb
# Or
nbdeck notebook.ipynb -o slides.ipynb
# Or
nbdeck notebook.ipynb > slides.ipynb

Unlike the shell command, the nbdeck Python function does not create a file on its own. To create a converted file, use the nbformat and pathlib libraries.

import nbformat
from nbless import nbconv, nbdeck

# Create HTML slides from notebook.ipynb in notebooks folder
nbformat.write(nbdeck("notebook.ipynb"), "slides.ipynb")
filename, contents = nbconv("slides.ipynb", "slides")
Path(filename).write_text(contents)

Executing a notebook with nbexec

The nbexec command runs the input notebook from top to bottom. If an out_file name is not provided, the executed notebook contents will be printed to the screen (stdout). This can be useful for previewing the output.

nbexec notebook.ipynb

You can create or overwrite a notebook file with the --out_file or -o flag or by redirecting the output to a file with >.

nbexec notebook.ipynb --out_file executed.ipynb
# Or
nbexec notebook.ipynb -o executed.ipynb
# Or
nbexec notebook.ipynb > executed.ipynb

The default kernel is python3, but it is possible to specify the kernel that will be used to run notebook with the --kernel or -k flag as in the example with the IRkernel below.

nbexec notebook.ipynb --kernel ir --out_file notebook.ipynb
# Or
nbexec notebook.ipynb -k ir -o notebook.ipynb

Unlike the shell command, the nbexec Python function does not create a file on its own. To create a notebook file, use the nbformat library.

import nbformat
from nbless import nbexec

# Create notebook.ipynb from notebook.ipynb
    nb = nbexec("notebook.ipynb")
nbformat.write(nb, "executed.ipynb")
    Rnb = nbexec("Rnotebook.ipynb")
nbformat.write(Rnb, "Rexecuted.ipynb", kernel="ir")

Creating and executing a Jupyter notebook with nbless

The nbless shell command executes a notebook created from code and markdown/text files.

nbless README.md plot.py notes.txt > executed.ipynb

The default kernel is python3, but it is possible to specify the kernel that will be used to run notebook with the --kernel or -k flag.

nbless README.md plot.py notes.txt --kernel ir > Rnotebook.ipynb
# Or
nbless README.md plot.py notes.txt -k ir > Rnotebook.ipynb

Instead of redirecting to a file (>), you can use the --out_file or -o flag:

nbless README.md plot.py notes.txt --out_file executed.ipynb
# Or
nbless README.md plot.py notes.txt -o executed.ipynb

Unlike the shell command, the nbless Python function does not create a file on its own. To create a notebook file, use the nbformat library.

import nbformat
from nbless import nbless

# Build and execute a notebook with nbless
    nb = nbless(["plot.py", "notes.txt"])
nbformat.write(nb, "executed.ipynb")
    Rnb = nbless(["plot.R", "notes.txt"], kernel="ir")
nbformat.write(Rnb, "Rexecuted.ipynb")

Extracting source files from a Jupyter notebook with nbraze

The nbraze shell command takes the contents of Jupyter Notebook code cells and turns them into code files, e.g. Python or R code files (.py or .R). The contents of markdown cells are turned into markdown files.

nbraze notebook.ipynb

The default code file extension for nbraze is py, but it is possible to set the file extension with the --extension or -e flag. If the language_info key is defined in the Jupyter notebook metadata, nbraze can try to infer the code file extension from the programming language.

nbraze notebook.ipynb --extension R
nbraze notebook.ipynb -e js

from nbless import nbraze

# Create source files from notebook.ipynb
    nbraze("notebook.ipynb")
    nbraze("notebook.ipynb", extension="R")

Creating a Jupyter notebook with nbuild

The nbuild shell command takes the contents of Python or R code files (.py or .R) and stores them as Jupyter Notebook code cells. The contents of all other files are stored in markdown cells.

nbuild README.md plot.py notes.txt > notebooks/notebook.ipynb

Instead of redirecting to a file (>), you can use the --out_file or -o flag:

nbuild README.md plot.py notes.txt --out_file notebooks/notebook.ipynb
# Or
nbuild README.md plot.py notes.txt -o notebooks/notebook.ipynb

You can preview the raw notebook output by running nbuild with only the positional arguments:

nbuild README.md plot.py notes.txt

The nbuild Python function does not create a file on its own. To create a notebook file, use the nbformat library.

import nbformat
from nbless import nbuild

# Create notebook.ipynb from plot.py and notes.txt
nb = nbuild(["plot.py", "notes.txt"])
nbformat.write(nb, "notebook.ipynb")

Related projects

• pandoc
• jupytext
• notedown

The packages listed above can all convert Jupyter notebooks to other formats:

• markdown files (all three) or
• Python scripts (jupytext).

Nbless wraps jupyter nbconvert to convert notebooks to other file types, but it can do something all of the aforementioned packages cannot. Nbless can take a more modular approach to file conversion by extracting the contents of each notebook cell into a separate file (cell -> file) or using a source file to create each notebook cell (file -> cell). Looking beyond simple file conversion, Nbless includes a tool for making slides from notebooks (by setting slide_type in notebook metadata).

Next Steps

Currently, notebook metadata is lost when using nbraze/nbuild/nbless.

• Enable nbuild/nbless to accept metadata via a metadata.json file.
• Enable nbraze to output metadata via a metadata.json file.