Why?

A need for literate programming

A personal note.

I use Jupyter notebooks on a daily basis as a part of my PhD analysis. Until very recently, I was following the relatively standard approach of building a pure-Python package 📦 which was imported into my notebooks 📓, such that the notebooks themselves remained simple and focused on analysis 🔬. Inevitably, this meant that a large portion of my code was unable to leverage the rich and dynamic interface offered by Jupyter notebooks. I would write tests ☑️ in a separate directory, and example notebooks ❓ in a notebooks directory, meaning that the tests, examples, and implementation were scattered across several locations. 😲

When I read the description of the nbdev repository, I immediately realised that notebook-based literate programming would solve a problem that I hadn't even realised that I had ✨. Putting the tests, examples, and code all in one place, with support for Markdown rendering and visualisations vastly improved the legibility and reproducibility of my code. 🍾

Creating my own library

Although it was the feature-list of nbdev that enabled me to see the need for such a project, I decided early on that I wanted something that targets existing Python package writers. I didn't want the generated Python source code to vastly differ from the notebooks; I wanted it to be relatively easy to understand how the final package layout is mapped to by the source notebooks. 🤔

Additionally, I wanted to have as little runtime involvement as possible; in particular to avoid asking end-users of the package to install the entire nbconvert toolchain. 🏋️

Also, realistically, this kind of project is fun :)

This post may have been sponsored by emojis