From 38db73a658a82fd49cbc644d2d0922068a44761a Mon Sep 17 00:00:00 2001 From: Charlie Marsh Date: Sun, 22 Sep 2024 13:56:01 -0400 Subject: [PATCH 1/2] Document uv-with-Jupyter workflows --- docs/guides/index.md | 1 + docs/guides/integration/jupyter.md | 76 ++++++++++++++++++++++++++++++ mkdocs.template.yml | 1 + 3 files changed, 78 insertions(+) create mode 100644 docs/guides/integration/jupyter.md diff --git a/docs/guides/index.md b/docs/guides/index.md index 07a6f5473b00..2a637898313a 100644 --- a/docs/guides/index.md +++ b/docs/guides/index.md @@ -10,6 +10,7 @@ Check out one of the core guides to get started: Learn how to integrate uv with other software: - [Using in Docker images](./integration/docker.md) +- [Using with Jupyter](./integration/jupyter.md) - [Using with pre-commit](./integration/pre-commit.md) - [Using in GitHub Actions](./integration/github.md) - [Using with alternative package indexes](./integration/alternative-indexes.md) diff --git a/docs/guides/integration/jupyter.md b/docs/guides/integration/jupyter.md new file mode 100644 index 000000000000..a3375459a15a --- /dev/null +++ b/docs/guides/integration/jupyter.md @@ -0,0 +1,76 @@ +# Using uv with Jupyter + +There are a few critical considerations: + +- Are you working with a project (in which case, you want `uv run`)? Or are you working with a + standalone notebook (in which case, you want `uvx`)? +- Do you need to install packages from within Jupyter? Or is the environment read-only? The latter + is way easier; the former requires some extra work. +- Are you trying to run Jupyter directly? Or through an editor, like VS Code? + +## As a standalone tool... + +If you're working within a [project](../../concepts/projects.md), you can kick off a Jupyter server +with access to the project's virtual environment via the following: + +```console +$ uv run --with jupyter jupyter notebook +``` + +Within the notebook, you can then import your project's modules as you would in any other `uv run` +invocation. For example, if your project depends on `requests`, `import requests` will import +`requests` from the project's virtual environment. + +While `jupyter` itself is installed in an isolated environment when used via +`uv run --with jupyter`, within the notebook, `!uv add` and related commands will modify the +_project's_ environment. + +For example, running `!uv add pydantic` from within a notebook will add `pydantic` to the project's +dependencies and virtual environment, such that `import pydantic` will work immediately, without +further configuration or a server restart. + +!!! note + + Since the Jupyter server is running in an isolated virtual environment, `!uv pip install` will install package's + into _Jupyter's_ environment, not the project environment. Such dependencies may disappear on subsequent `jupyter` + invocations. To install packages into the project environment, use `!uv add`. + +If you're working with a notebook that relies on pip (e.g., via the `%pip` magic), you can include +pip in your project's virtual environment by running `uv venv --seed` prior to starting the Jupyter +server. For example, given: + +```console +$ uv venv --seed +$ uv run --with jupyter jupyter notebook +``` + +Subsequent `%pip install` invocations within the notebook will install packages into the project's +virtual environment. However, such modifications will _not_ be reflected in the project's +`pyproject.toml` or `uv.lock` files. + +## As a project dependency... + +## Within VS Code + +```console +# Create a new notebook project +# and open the directory in code +$ uv init my-notebook +$ cd my-notebook +$ uv add ipykernel +$ code . +``` + +- Now that the new project directory is open in code, use the action "Create: New Jupyter Notebook" +- Click Select Kernel -> Python Environments +- Select the virtual environment that uv created. It will be named .venv/bin/python in this dropdown + (or maybe .venv\Scripts\python on windows) + +If you don't `uv add ipykernel`, the notebook will fail to execute with an error. + +## Notes + +- If you run `uv add --dev ipykernel`, then `uv run ipython kernel install --user --name=uv`, you + can then run within the project environment even if Jupyter is installed elsewhere. This is great, + because `!uv pip install` will install packages into the project environment, not the Jupyter + environment. The downside is you need to add these as dev dependencies. diff --git a/mkdocs.template.yml b/mkdocs.template.yml index cf14bd2bafe2..60461b2540e3 100644 --- a/mkdocs.template.yml +++ b/mkdocs.template.yml @@ -110,6 +110,7 @@ nav: - Integration guides: - guides/integration/index.md - Docker: guides/integration/docker.md + - Jupyter: guides/integration/jupyter.md - GitHub Actions: guides/integration/github.md - Pre-commit: guides/integration/pre-commit.md - FastAPI: guides/integration/fastapi.md From 3352698d39360e33631344ae90dad0a3ab0e215c Mon Sep 17 00:00:00 2001 From: Charlie Marsh Date: Mon, 23 Sep 2024 14:26:46 -0400 Subject: [PATCH 2/2] Second round --- docs/guides/integration/jupyter.md | 149 +++++++++++++++++++++-------- 1 file changed, 111 insertions(+), 38 deletions(-) diff --git a/docs/guides/integration/jupyter.md b/docs/guides/integration/jupyter.md index a3375459a15a..dbb5b267f768 100644 --- a/docs/guides/integration/jupyter.md +++ b/docs/guides/integration/jupyter.md @@ -1,39 +1,78 @@ # Using uv with Jupyter -There are a few critical considerations: +The [Jupyter](https://jupyter.org/) notebook is a popular tool for interactive computing, data +analysis, and visualization. You can use Jupyter with uv in a few different ways, either to interact +with a project, or as a standalone tool. -- Are you working with a project (in which case, you want `uv run`)? Or are you working with a - standalone notebook (in which case, you want `uvx`)? -- Do you need to install packages from within Jupyter? Or is the environment read-only? The latter - is way easier; the former requires some extra work. -- Are you trying to run Jupyter directly? Or through an editor, like VS Code? +## Using Jupyter within a project -## As a standalone tool... - -If you're working within a [project](../../concepts/projects.md), you can kick off a Jupyter server +If you're working within a [project](../../concepts/projects.md), you can start a Jupyter server with access to the project's virtual environment via the following: ```console -$ uv run --with jupyter jupyter notebook +$ uv run --with jupyter jupyter lab ``` -Within the notebook, you can then import your project's modules as you would in any other `uv run` -invocation. For example, if your project depends on `requests`, `import requests` will import +By default, `jupyter lab` will start the server at +[http://localhost:8888/lab](http://localhost:8888/lab). + +Within a notebook, you can import your project's modules as you would in any other file in the +project. For example, if your project depends on `requests`, `import requests` will import `requests` from the project's virtual environment. -While `jupyter` itself is installed in an isolated environment when used via -`uv run --with jupyter`, within the notebook, `!uv add` and related commands will modify the -_project's_ environment. +If you're looking for read-only access to the project's virtual environment, then there's nothing +more to it. However, if you need to install additional packages from within the notebook, there are +a few extra details to consider. + +### Creating a kernel + +If you need to install packages from within the notebook, we recommend creating a dedicated kernel +for your project. Kernels enable the Jupyter server to run in one environment, with individual +notebooks running in their own, separate environments. + +In the context of uv, we can create a kernel for a project while installing Jupyter itself in an +isolated environment, as in `uv run --with jupyter jupyter lab`. Creating a kernel for the project +ensures that the notebook is hooked up to the correct environment, and that any packages installed +from within the notebook are installed into the project's virtual environment. + +To create a kernel, you'll need to install `ipykernel` as a development dependency: + +```console +$ uv add --dev ipykernel +``` + +Then, you can create the kernel for `project` with: + +```console +$ uv run ipython kernel install --user --name=project +``` + +From there, start the server with: + +```console +$ uv run --with jupyter jupyter lab +``` + +When creating a notebook, select the `project` kernel from the dropdown. Then use `!uv add pydantic` +to add `pydantic` to the project's dependencies, or `!uv pip install pydantic` to install `pydantic` +into the project's virtual environment without persisting the change to the project `pyproject.toml` +or `uv.lock` files. Either command will make `import pydantic` work within the notebook. + +### Installing packages without a kernel + +If you don't want to create a kernel, you can still install packages from within the notebook. +However, there are a few caveats to consider. + +Though `uv run --with jupyter` runs in an isolated environment, within the notebook itself, +`!uv add` and related commands will modify the _project's_ environment, even without a kernel. For example, running `!uv add pydantic` from within a notebook will add `pydantic` to the project's dependencies and virtual environment, such that `import pydantic` will work immediately, without further configuration or a server restart. -!!! note - - Since the Jupyter server is running in an isolated virtual environment, `!uv pip install` will install package's - into _Jupyter's_ environment, not the project environment. Such dependencies may disappear on subsequent `jupyter` - invocations. To install packages into the project environment, use `!uv add`. +However, since the Jupyter server is the "active" environment, `!uv pip install` will install +package's into _Jupyter's_ environment, not the project environment. Such dependencies will persist +for the lifetime of the Jupyter server, but may disappear on subsequent `jupyter` invocations. If you're working with a notebook that relies on pip (e.g., via the `%pip` magic), you can include pip in your project's virtual environment by running `uv venv --seed` prior to starting the Jupyter @@ -41,36 +80,70 @@ server. For example, given: ```console $ uv venv --seed -$ uv run --with jupyter jupyter notebook +$ uv run --with jupyter jupyter lab ``` Subsequent `%pip install` invocations within the notebook will install packages into the project's virtual environment. However, such modifications will _not_ be reflected in the project's `pyproject.toml` or `uv.lock` files. -## As a project dependency... +## Using Jupyter as a standalone tool + +If you ever need ad hoc access to a notebook (i.e., to run a Python snippet interactively), you can +start a Jupyter server at any time with `uv tool run jupyter lab`. This will run a Jupyter server in +an isolated environment. + +## Using Jupyter with a non-project environment + +If you need to run Jupyter in a virtual environment that isn't associated with a +[project](../../concepts/projects.md) (e.g., has no `pyproject.toml` or `uv.lock`), you can do so by +adding Jupyter to the environment directly. For example: + +```console +$ uv venv --seed +$ uv pip install pydantic +$ uv pip install jupyterlab +$ .venv/bin/jupyter lab +``` + +From here, `import pydantic` will work within the notebook, and you can install additional packages +via `!uv pip install`, or even `!pip install`. -## Within VS Code +## Using Jupyter from VS Code + +You can also engage with Jupyter notebooks from within an editor like VS Code. To connect a +uv-managed project to a Jupyter notebook within VS Code, we recommend creating a kernel for the +project, as in the following: ```console -# Create a new notebook project -# and open the directory in code -$ uv init my-notebook -$ cd my-notebook -$ uv add ipykernel +# Create a project. +$ uv init project +# Move into the project directory. +$ cd project +# Add ipykernel as a dev dependency. +$ uv add --dev ipykernel +# Open the project in VS Code. $ code . ``` -- Now that the new project directory is open in code, use the action "Create: New Jupyter Notebook" -- Click Select Kernel -> Python Environments -- Select the virtual environment that uv created. It will be named .venv/bin/python in this dropdown - (or maybe .venv\Scripts\python on windows) +Once the project directory is open in VS Code, you can create a new Jupyter notebook by selecting +"Create: New Jupyter Notebook" from the command palette. When prompted to select a kernel, choose +"Python Environments" and select the virtual environment you created earlier (e.g., +`.venv/bin/python`). -If you don't `uv add ipykernel`, the notebook will fail to execute with an error. +!!! note -## Notes + VS Code requires `ipykernel` to be present in the project environment. If you'd prefer to avoid + adding `ipykernel` as a dev dependency, you can install it directly into the project environment + with `uv pip install ipykernel`. + +If you need to manipulate the project's environment from within the notebook, you may need to add +`uv` as an explicit development dependency: + +```console +$ uv add --dev uv +``` -- If you run `uv add --dev ipykernel`, then `uv run ipython kernel install --user --name=uv`, you - can then run within the project environment even if Jupyter is installed elsewhere. This is great, - because `!uv pip install` will install packages into the project environment, not the Jupyter - environment. The downside is you need to add these as dev dependencies. +From there, you can use `!uv add pydantic` to add `pydantic` to the project's dependencies, or +`!uv pip install pydantic` to install `pydantic` into the project's virtual environment without +updating the project's `pyproject.toml` or `uv.lock` files.