Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enh(dev): Make default environment the developer environment #6456

Open
wants to merge 20 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 75 additions & 17 deletions doc/developer_guide/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,9 @@ To contribute to HoloViews, you will also need [Github account](https://github.c

### Pixi

Developing all aspects of HoloViews requires a wide range of packages in different environments. To make this more manageable, Pixi manages the developer experience. To install Pixi, follow [this guide](https://pixi.sh/latest/#installation).
Developing all aspects of HoloViews requires a wide range of packages in different environments, but for new contributors the `default` environment will be more than enough.

To make this more manageable, Pixi manages the developer experience. To install Pixi, follow [this guide](https://pixi.sh/latest/#installation).

#### Glossary

Expand All @@ -43,6 +45,7 @@ For more information, see the [Pixi documentation](https://pixi.sh/latest/).
The first time you run `pixi`, it will create a `.pixi` directory in the source directory.
This directory will contain all the files needed for the virtual environments.
The `.pixi` directory can be large, so it is advised not to put the source directory into a cloud-synced directory.

:::

## Installing the Project
Expand All @@ -60,13 +63,30 @@ This `holoviews` directory is the _source checkout_ for the remainder of this do

## Start developing

To start developing, run the following command
To start developing, run the following command, this will create an environment called `default` and install HoloViews in [editable mode](https://pip.pypa.io/en/stable/topics/local-project-installs/#editable-installs):

```bash
pixi run install
```

:::{admonition} Note
:class: info

The first time you run it, it will create a `pixi.lock` file with information for all available environments.
This command will take a minute or so to run.

:::
:::{admonition} Advanced usage
:class: tip

Currently, an editable install needs to be run in each environment. So, if you want to install in the `test-core` environment, you can add `--environment` / `-e` to the command:

```bash
pixi install
pixi run -e test-core install
```

The first time you run it, it will create a `pixi.lock` file with information for all available environments. This command will take a minute or so to run.
:::

When this is finished, it is possible to run the following command to download the data HoloViews tests and examples depend upon.

```bash
Expand All @@ -85,24 +105,41 @@ Syncing the git tags can be done with:
pixi run sync-git-tags
```

### Editable install
## Developer Environment

It can be advantageous to install the HoloViews in [editable mode](https://pip.pypa.io/en/stable/topics/local-project-installs/#editable-installs):
The `default` environment is meant to provide all the tools needed to develop HoloViews.

```bash
pixi run install
```
This environment can be created by running `pixi run install`, which will set up the environment and make an editable install of HoloViews.
Comment on lines +110 to +112
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs already mention pixi run install, that feels a little redundant. I'd just remove it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is ok to repeat ourselves, especially when it is essential to getting started.


:::{admonition} Note
:class: info
To activate this environment you can run `pixi shell`, this is equivalent to `source venv/bin/activate` in a virtual environment or `conda activate` in a conda environment.

Currently, this needs to be run for each environment. So, if you want to install in the `test-ui` environment, you can add `--environment` / `-e` to the command:
If you need to run a command directly instead of via `pixi`, you activate the environment and run your command (e.g. `pytest holoviews/tests/<somefile.py>`).

```bash
pixi run -e test-ui install
```
### VS Code

:::
This environment can also be selected in your IDE. In VS Code, this can be done by running the command `Python: Select Interpreter` and choosing `{'default': Pixi}`.

<p style="text-align: center">
<img
src="https://assets.holoviews.org/static/dev_guide/001.png"
alt="001"
style="width: 45%; display: inline-block"
/>
<img
src="https://assets.holoviews.org/static/dev_guide/002.png"
alt="002"
style="width: 45%; display: inline-block"
/>
</p>

To confirm you are using this dev environment, check the bottom right corner:

![003](https://assets.holoviews.org/static/dev_guide/003.png)

### Jupyter Lab

You can launch Jupyter lab with the `default` environment with `pixi run lab`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like we should let contributors know that they should run this command when:

  • they need to edit the docs (as building the docs is usually overkill)
  • they need to debug example tests

Either here, or perhaps preferably in the relevant sections.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added some extra text.

This can be advantageous when you need to edit the documentation or debug an example notebook.

## Linting

Expand All @@ -118,6 +155,18 @@ Linting can also be set up to run automatically with each commit; this is the re
pixi run lint-install
```

:::{admonition} Note
:class: info

Alternatively, if you have `pre-commit` installed elsewhere you can run

```bash
pre-commit install # To install
pre-commit run --all-files # To run on all files
```

:::

## Testing

To help keep HoloViews maintainable, all Pull Requests (PR) with code changes should typically be accompanied by relevant tests. While exceptions may be made for specific circumstances, the default assumption should be that a Pull Request without tests will not be merged.
Expand All @@ -133,9 +182,18 @@ Unit tests can be run with the `test-unit` task:
pixi run test-unit
```

:::{admonition} Advanced usage
:class: tip

The task is available in the following environments: `test-39`, `test-310`, `test-311`, `test-312`, and `test-core`. Where the first ones have the same environments except for different Python versions, and `test-core` only has a core set of dependencies.

If you haven't set the environment flag in the command, a menu will help you select which one of the environments to use.
You can run the task in a specific environment with the `-e` flag. For example, to run the `test-unit` task in the `test-39` environment, you can run:

```bash
pixi run -e test-39 test-unit
```

:::

### Example tests

Expand Down
130 changes: 83 additions & 47 deletions pixi.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,28 +3,60 @@ name = "holoviews"
channels = ["pyviz/label/dev", "bokeh/label/rc", "conda-forge"]
platforms = ["linux-64", "osx-arm64", "osx-64", "win-64"]

[tasks]
check-latest-packages = 'python scripts/check_latest_packages.py'
download-data = 'python scripts/download_data.py'
install = 'python -m pip install --no-deps --disable-pip-version-check -e .'
sync-git-tags = 'python scripts/sync_git_tags.py holoviews'

[activation.env]
PYTHONIOENCODING = "utf-8"

[environments]
test-39 = ["py39", "test-core", "test", "example", "test-example", "test-unit-task"]
test-310 = ["py310", "test-core", "test", "example", "test-example", "test-unit-task"]
test-311 = ["py311", "test-core", "test", "example", "test-example", "test-unit-task"]
test-312 = ["py312", "test-core", "test", "example", "test-example", "test-unit-task"]
test-ui = ["py312", "test-core", "test", "test-ui"]
test-core = ["py313", "test-core", "test-unit-task"]
test-gpu = ["py312", "test-core", "test", "test-gpu"]
docs = ["py311", "example", "doc"]
build = ["py311", "build"]
lint = ["py311", "lint"]

[dependencies]
default = [
"required",
"py312",
"optional",
"test-core",
"test-example",
"test-unit-task",
"test-ui",
"lint",
"dev",
]

[environments.test-39]
features = ["required", "py39", "optional", "test-core", "test-example", "test-unit-task"]
no-default-feature = true

[environments.test-310]
features = ["required", "py310", "optional", "test-core", "test-example", "test-unit-task"]
no-default-feature = true

[environments.test-311]
features = ["required", "py311", "optional", "test-core", "test-example", "test-unit-task"]
no-default-feature = true

[environments.test-312]
features = ["required", "py312", "optional", "test-core", "test-example", "test-unit-task"]
no-default-feature = true

[environments.test-ui]
features = ["required", "py312", "optional", "test-core", "test-ui"]
no-default-feature = true

[environments.test-core]
features = ["required", "py313", "test-core", "test-unit-task"]
no-default-feature = true

[environments.test-gpu]
features = ["required", "py312", "test-core", "optional", "test-gpu"]
no-default-feature = true

[environments.docs]
features = ["required", "py311", "optional", "doc"]
no-default-feature = true

[environments.build]
features = ["required", "py311", "build"]
no-default-feature = true

[environments.lint]
features = ["lint"]
no-default-feature = true

[feature.required.dependencies]
nomkl = "*"
pip = "*"
# Required
Expand All @@ -37,6 +69,15 @@ panel = ">=1.0"
param = ">=2.0,<3.0"
pyviz_comms = ">=2.1"

[feature.required.tasks]
check-latest-packages = 'python scripts/check_latest_packages.py'
download-data = 'python scripts/download_data.py'
install = 'python -m pip install --no-deps --disable-pip-version-check -e .'
sync-git-tags = 'python scripts/sync_git_tags.py holoviews'

[feature.required.activation.env]
PYTHONIOENCODING = "utf-8"

[feature.py39.dependencies]
python = "3.9.*"
panel = "1.4.*"
Expand All @@ -57,13 +98,17 @@ bokeh_sampledata = "*"
python = "3.13.*"
bokeh_sampledata = "*"

[feature.example.dependencies]
[feature.optional.dependencies]
cftime = "*"
contourpy = "*"
dask-core = "*"
dask-expr = "*"
datashader = ">=0.11.1"
ffmpeg = "*"
ibis-sqlite = "*"
ipython = ">=5.4.0"
matplotlib-base = ">=3"
nbconvert-core = "*"
netcdf4 = "*"
networkx = "*"
notebook = "*"
Expand All @@ -72,10 +117,24 @@ plotly = ">=4.0"
pooch = "*"
pyarrow = "*"
scikit-image = "*"
scipy = "*"
scipy = ">=1.10" # Python 3.9 + Windows downloads 1.9
selenium = "*"
shapely = "*"
spatialpandas = "*"
streamz = ">=0.5.0"
xarray = ">=0.10.4"
xyzservices = "*"

[feature.optional.target.unix.dependencies]
tsdownsample = "*" # currently not available on Windows

[feature.dev.dependencies]
jupyterlab = "*"
jupyterlab-myst = "*"
setuptools_scm = "*"

[feature.dev.tasks]
lab = 'jupyter lab'

# =============================================
# =================== TESTS ===================
Expand All @@ -89,32 +148,9 @@ pytest-github-actions-annotate-failures = "*"
pytest-rerunfailures = "*"
pytest-xdist = "*"

[feature.test-unit-task.tasks] # So it is not showing up in the test-ui environment
[feature.test-unit-task.tasks] # So it is not showing up in the test-gpu + test-ui environment
test-unit = 'pytest holoviews/tests -n logical --dist loadgroup'

[feature.test.dependencies]
cftime = "*"
contourpy = "*"
dask-core = "*"
dask-expr = "*"
datashader = ">=0.11.1"
ffmpeg = "*"
ibis-sqlite = "*"
nbconvert = "*"
matplotlib-base = ">=3"
networkx = "*"
plotly = ">=4.0"
pillow = "*"
scipy = ">=1.10" # Python 3.9 + Windows downloads 1.9
selenium = "*"
shapely = "*"
spatialpandas = "*"
xarray = ">=0.10.4"
xyzservices = "*"

[feature.test.target.unix.dependencies]
tsdownsample = "*" # currently not available on Windows

[feature.test-example.tasks]
test-example = { cmd = 'pytest -n logical --dist loadscope --nbval-lax examples', env = { DASK_DATAFRAME__QUERY_PLANNING = "False" } }

Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,7 @@ filterwarnings = [
"ignore::ResourceWarning",
# 2024-11
"ignore:The legacy Dask DataFrame implementation is deprecated:FutureWarning", # https://github.com/holoviz/spatialpandas/issues/146
"ignore:The cuda.cuda(rt)? module is deprecated:DeprecationWarning", # https://github.com/rapidsai/rmm/issues/1730
]

[tool.coverage]
Expand Down