Use hatch backend #6425

blink1073 · 2022-05-14T20:34:58Z

Uses new hatch_jupyter_builder plugin
hatch is a newly minted pypa project, and appears to be a drop-in replacement for our use of setuptools / jupyter-packaging

github-actions · 2022-05-14T20:35:10Z

👈 Launch a Binder on branch blink1073/notebook/use-hatch

blink1073 · 2022-05-15T02:03:30Z

We're blocked by pypa/hatch#243.

blink1073 · 2022-05-15T03:32:27Z

Found a workaround using force-include

ofek · 2022-05-18T00:09:56Z

FYI just released https://github.com/pypa/hatch/releases/tag/hatchling-v1.0.0

Also I think you can copy what you did here ipython/ipyparallel@e3e896a

blink1073 · 2022-05-18T00:13:37Z

Thanks, and congratulations!

blink1073 · 2022-05-18T02:28:00Z

We can remove setup.py once jupyter-server/jupyter_releaser#319 is released, then this should be good to go!

ofek · 2022-05-18T02:57:01Z

It's undocumented until I find an auto-generator for non-Click CLIs, but if build deps are met you can do hatchling version

blink1073 · 2022-05-18T10:53:23Z

if build deps are met you can do hatchling version

Thanks! Perhaps https://sphinx-argparse.readthedocs.io/en/stable/?

blink1073 · 2022-05-18T11:00:35Z

Perhaps sphinx-argparse.readthedocs.io/en/stable?

Ah, you are using mkdocs...

blink1073 · 2022-05-18T13:26:16Z

@jtpio, any idea why the snapshot tests are failing?

jtpio · 2022-05-18T14:03:08Z

It looks like the menu and toolbar customization files are missing (defined in schema) for one of the failures:

blink1073 · 2022-05-18T14:27:15Z

It looks like the menu and toolbar customization files are missing (defined in schema) for one of the failures

Thanks!

blink1073 · 2022-05-18T16:50:57Z

Hmm, after adding the schemas the tests don't appear to start at all.

jtpio · 2022-05-20T07:11:00Z

Hmm, after adding the schemas the tests don't appear to start at all.

hmm strange. For the UI tests the package is first installed with the following:

python -m pip install -vv notebook*.whl

Will check locally.

jtpio · 2022-05-20T17:00:23Z

Nice, CI is now passing with fcdb15f.

Looks like there are a few conflicts after the merge of #6429.

add flake8 config add artifacts cleanup use tbump to get current version cleanup for new dep versions clean up workflows switch back to bumpversion fix verion fixup fixup fixup fixup fix version check fixups try version handling again version cleanup fix typescript error undo bump2version changes include schemas and add build timeouts fix workflow syntax more workflow cleanup clean up config

jtpio · 2022-05-31T07:11:09Z

Ah I thought CI was already uploading the built assets as artifacts.

Maybe we could add the following step to the check release workflow so it's easier to inspect the sdist and wheel?

- name: Upload Distributions
  uses: actions/upload-artifact@v2
  with:
    name: notebook-jupyter-releaser-dist-${{ github.run_number }}
    path: .jupyter_releaser_checkout/dist

blink1073 · 2022-05-31T16:12:09Z

Good call. I also played with my wip comparison script and got this output now:

python ../extension-cookiecutter-ts/compare.py ../notebook_orig . sdist

Removed_files:
notebook-7.0.0a4/jupyter-config/jupyter_server_config.d
notebook-7.0.0a4/jupyter-config/jupyter_server_config.d/notebook.json
notebook-7.0.0a4/notebook/labextension/static/remoteEntry.4fb60df0e7df1388ab11.js


Added files:
notebook-7.0.0a4/notebook/labextension/static/remoteEntry.5ea5fe7dc5a5fbc91231.js
notebook-7.0.0a4/jupyter_config.json
notebook-7.0.0a4/lerna.json
notebook-7.0.0a4/pyrightconfig.json
notebook-7.0.0a4/.eslintrc.json

blink1073 · 2022-05-31T19:15:20Z

Here's the latest output:

Successfully built notebook-7.0.0a4.tar.gz
Removed_files:
notebook-7.0.0a4/jupyter-config/jupyter_server_config.d
notebook-7.0.0a4/notebook/labextension/static/remoteEntry.4fb60df0e7df1388ab11.js


Added files:
notebook-7.0.0a4/notebook/labextension/static/remoteEntry.5ea5fe7dc5a5fbc91231.js

Successfully built notebook-7.0.0a4-py3-none-any.whl
Removed_files:
notebook/labextension/static/remoteEntry.4fb60df0e7df1388ab11.js
notebook-7.0.0a4.data/data/share/jupyter/labextensions/@jupyter-notebook/lab-extension/static/remoteEntry.4fb60df0e7df1388ab11.js
notebook-7.0.0a4.dist-info/top_level.txt

Added files:
notebook-7.0.0a4.data/data/share/jupyter/labextensions/@jupyter-notebook/lab-extension/static/remoteEntry.5ea5fe7dc5a5fbc91231.js
notebook/labextension/static/remoteEntry.5ea5fe7dc5a5fbc91231.js

jtpio · 2022-06-01T08:04:47Z

Looking good!

jtpio · 2022-06-01T08:14:52Z

The Binder for this PR fails to build because of the following error:

Which seems to be related to the jupyter labextension develop command expecting to find a setup.py:

https://github.com/jupyterlab/jupyterlab/blob/6099bcb4d32902bd61b82df818cdd2df3fe1685c/jupyterlab/federated_labextensions.py#L427-L438

jtpio · 2022-06-01T12:31:34Z

Which seems to be related to the jupyter labextension develop command expecting to find a setup.py:

This is likely to be fixed by jupyterlab/jupyterlab#12606.

blink1073 · 2022-06-01T12:42:44Z

Yeah I think we need to use a setup.py shim for extensions that are targeting versions of jupyterlab that require it. Once we have 3.x and 4.x versions that don't require it, the cookiecutter itself can be updated to not include it. For the migration script, we can include it if there was already one there and eventually remove it.

blink1073 · 2022-06-01T14:26:44Z

Drat, now we have the issue that symlinks can't be created over existing files. With setuptools the shared data was not written in editable mode. It feels like we need to fix a few bugs and backport them in JupyterLab before doing any migrations. We shouldn't need a setup.py and we shouldn't error out when overwriting symlinks.

ofek · 2022-06-01T15:00:59Z

Hatchling resolves symlinks fyi

blink1073 · 2022-06-01T15:03:16Z

The issue is the JupyterLab build command is trying to create symlinks over the files that are placed into /share and causing an error. This issue is purely of our own making. 😄

blink1073 · 2022-06-10T22:18:29Z

Ah, the symlink error was actually from a file in this repository, the binder is working now. I actually think this PR is good to go now.

agoose77 · 2022-06-10T22:37:56Z

pyproject.toml

+source = "code"
+
+[tool.hatch.build.targets.wheel.shared-data]
+"notebook/labextension" = "share/jupyter/labextensions/@jupyter-notebook/lab-extension"


@blink1073 I'm not an expert on the filesystem layout for JupyterLab extensions, but should this be

Suggested change

"notebook/labextension" = "share/jupyter/labextensions/@jupyter-notebook/lab-extension"

"notebook/labextension" = "share/jupyter/labextensions/@jupyter-notebook"

Looks like this should be kept since it maps to "notebook/labextension" on the left hand side?

Yes, I agree

Hmm, I could be wrong but I thought the output directory contents were supposed to end up in share/jupyter/labextensions/@jupyter-notebook/

If you look e.g. at the wheel for jupyterlab-katex, the contents of /jupyterlab_katex-3.3.0.data/data/share/jupyter/labextensions/@jupyterlab/katex-extension/ is

schemas static install.json package.json

whereas in notebook's wheel we currently have the following files in /notebook-7.0.0a4.data/data/share/jupyter/labextensions/@jupyter-notebook/:

lab-extension

E.g. this is jupyterlab-myst which works: https://github.com/agoose77/jupyterlab-myst/blob/320b15187ee605b1f948619e9e1f9cfa672eaf00/pyproject.toml#L57

The extension is nested in our case, under @jupyter-notebook. The full extension name is @jupyter-notebook/lab-extension.

Right, I'm with you. This is where the fact that / is allowed in NPM package names threw me for a second!

jtpio · 2022-06-13T12:11:40Z

jupyter_config.json

@@ -1,4 +0,0 @@
-{
-  "LabApp": { "collaborative": true, "expose_app_in_browser": true },
-  "JupyterNotebookApp": { "collaborative": true, "expose_app_in_browser": true }


Probably we want to keep this file, so RTC can be tested on Binder?

jtpio

Thanks!!

Looking good on Binder and also locally using the assets built by the releaser.

Left a small comment about the config for testing RTC on Binder, but otherwise looks good!

jtpio · 2022-06-13T15:51:43Z

Starting a new pre-release with this change.

jtpio · 2022-06-13T15:52:00Z

Ah looks like something is wrong with the Python version: #6448

blink1073 added the maintenance label May 14, 2022

blink1073 force-pushed the use-hatch branch from 02364fc to d0a6fe0 Compare May 14, 2022 20:38

blink1073 mentioned this pull request May 14, 2022

Optional dependencies not handled in migration script pypa/hatch#241

Closed

blink1073 force-pushed the use-hatch branch from fc97357 to 1c7fc5d Compare May 15, 2022 02:03

blink1073 mentioned this pull request May 15, 2022

How to forcibly include a directory? pypa/hatch#243

Closed

This was referenced May 16, 2022

Evaluation install fails since the recent flit migration jupyter/nbconvert#1776

Closed

Feat: move to hatchling? jupyter/jupyter-packaging#145

Closed

blink1073 force-pushed the use-hatch branch from 62cf256 to d33ade5 Compare May 18, 2022 02:07

jtpio added this to the 7.0 milestone May 18, 2022

blink1073 mentioned this pull request May 18, 2022

Move jupyter_builder_hatch plugin to jupyterlab org jupyterlab/frontends-team-compass#146

Closed

fcollonval mentioned this pull request May 19, 2022

Weekly Team Meetings: Jan-Jun 2022 jupyterlab/frontends-team-compass#135

Closed

afshin mentioned this pull request May 19, 2022

Notebook Weekly Meetings Nov 2021 - Dec 2022 jupyter/notebook-team-compass#5

Closed

blink1073 force-pushed the use-hatch branch from fcdb15f to cded2be Compare May 20, 2022 20:55

fix workflow

15e678f

update included files and upload dists

30360e0

more inclusion updates

b5b0b24

remove redundant files in wheel

44e35e4

jtpio mentioned this pull request Jun 1, 2022

Switch to hatch backend jupyterlab/jupyterlab#12606

Merged

11 tasks

blink1073 added 3 commits June 1, 2022 07:50

add setup.py shim

d6d3c45

skip check-manifest

cdc4ac1

Merge branch 'use-hatch' of github.com:blink1073/notebook into use-hatch

6bbfab3

Zsailer mentioned this pull request Jun 9, 2022

Meeting Notes 2022 jupyter-server/team-compass#15

Closed

fix unlinking

17c8c2a

blink1073 marked this pull request as ready for review June 10, 2022 22:18

agoose77 reviewed Jun 10, 2022

View reviewed changes

jtpio reviewed Jun 13, 2022

View reviewed changes

jtpio approved these changes Jun 13, 2022

View reviewed changes

restore binder config file

c0be687

blink1073 merged commit 614e478 into jupyter:main Jun 13, 2022

blink1073 deleted the use-hatch branch June 13, 2022 15:19

github-actions bot locked as resolved and limited conversation to collaborators Jun 14, 2023

	"notebook/labextension" = "share/jupyter/labextensions/@jupyter-notebook/lab-extension"
	"notebook/labextension" = "share/jupyter/labextensions/@jupyter-notebook"

Use hatch backend #6425

Use hatch backend #6425

Conversation

blink1073 commented May 14, 2022 • edited Loading

github-actions bot commented May 14, 2022

blink1073 commented May 15, 2022

blink1073 commented May 15, 2022

ofek commented May 18, 2022

blink1073 commented May 18, 2022

blink1073 commented May 18, 2022

ofek commented May 18, 2022

blink1073 commented May 18, 2022

blink1073 commented May 18, 2022

blink1073 commented May 18, 2022

jtpio commented May 18, 2022

blink1073 commented May 18, 2022

blink1073 commented May 18, 2022

jtpio commented May 20, 2022

jtpio commented May 20, 2022

jtpio commented May 31, 2022 • edited Loading

blink1073 commented May 31, 2022 • edited Loading

blink1073 commented May 31, 2022 • edited Loading

jtpio commented Jun 1, 2022

jtpio commented Jun 1, 2022

jtpio commented Jun 1, 2022

blink1073 commented Jun 1, 2022

blink1073 commented Jun 1, 2022

ofek commented Jun 1, 2022

blink1073 commented Jun 1, 2022

blink1073 commented Jun 10, 2022

agoose77 Jun 10, 2022

Choose a reason for hiding this comment

jtpio Jun 13, 2022

Choose a reason for hiding this comment

blink1073 Jun 13, 2022

Choose a reason for hiding this comment

agoose77 Jun 13, 2022 • edited Loading

Choose a reason for hiding this comment

blink1073 Jun 13, 2022

Choose a reason for hiding this comment

agoose77 Jun 13, 2022

Choose a reason for hiding this comment

jtpio Jun 13, 2022

Choose a reason for hiding this comment

blink1073 Jun 13, 2022

Choose a reason for hiding this comment

jtpio left a comment • edited Loading

Choose a reason for hiding this comment

jtpio commented Jun 13, 2022

jtpio commented Jun 13, 2022

blink1073 commented May 14, 2022 •

edited

Loading

jtpio commented May 31, 2022 •

edited

Loading

blink1073 commented May 31, 2022 •

edited

Loading

blink1073 commented May 31, 2022 •

edited

Loading

agoose77 Jun 13, 2022 •

edited

Loading

jtpio left a comment •

edited

Loading