Support package release dependencies in multi-Python packages

[davidbrochart]

Closes #371.

This fails with check-python, extract-release... because the dependent packages have not been released on PyPI yet.
Maybe there is a way to add the just-created (local) releases in the package providers?

[blink1073]

Yeah, I finally figured out how to test local changes in #363. Perhaps we should wait for that before doing this?

[davidbrochart]

Perhaps we should wait for that before doing this?

Sounds good. In the meantime, I can always achieve what this PR does in two steps:

• make a first release with version x.y.z.
• pin sub-packages with ==x.y.z, and release again.

[blink1073]

Okay, you can rebase now that #363 is merged.

[codecov-commenter]

Codecov Report

Merging #372 (2651a46) into main (2cd3e74) will decrease coverage by 0.32%.
The diff coverage is 58.33%.

@@            Coverage Diff             @@
##             main     #372      +/-   ##
==========================================
- Coverage   82.93%   82.60%   -0.33%     
==========================================
  Files          21       21              
  Lines        2866     2898      +32     
  Branches      374      382       +8     
==========================================
+ Hits         2377     2394      +17     
- Misses        356      368      +12     
- Partials      133      136       +3     

Impacted Files	Coverage Δ
jupyter_releaser/lib.py	78.68% <51.61%> (-2.15%)	⬇️
jupyter_releaser/cli.py	91.00% <100.00%> (+0.09%)	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[davidbrochart]

I rebased an tried to release jupyverse with this branch, but it failed in check-changelog with:

fatal: malformed object name refs/heads/release_deps

Any idea?

[blink1073]

Ah, I see that we need to store the RH_REF in the metadata file and use that.

[blink1073]

#375

[davidbrochart]

I rebased on #375, but it still fails.

[davidbrochart]

With the latest changes in #375, it fails at "Publish Release" in "Prepare Environment":

ValueError: No release found for url https://github.com/jupyter-server/jupyverse/releases/tag/untagged-3ab212a9b6e7bceff24f

But it could be because of my changes.

[blink1073]

I think it is because the original draft release was over one day old and got pruned. You'll have to start over.

[blink1073]

Oh, I see it is still there. Maybe try with the full url: https://github.com/jupyter-server/jupyverse/releases/tag/untagged-43adf70df0043151e18b

[davidbrochart]

Still fails:

--------------------------------------------------
bump-version
--------------------------------------------------


/home/runner/work/jupyter_releaser/jupyter_releaser/jupyter_releaser/util.py:174: UserWarning: Using deprecated setup.py invocation
  warnings.warn("Using deprecated setup.py invocation")
0.0.39
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.6/x64/bin/tbump", line 8, in <module>
    sys.exit(main())
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/tbump/cli.py", line 245, in main
    run(args)
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/tbump/cli.py", line 141, in run
    run_bump(arguments)
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/tbump/cli.py", line 162, in run_bump
    bump(bump_options, arguments.operations)
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/tbump/cli.py", line 172, in bump
    config_file = get_config_file(
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/tbump/config.py", line 222, in get_config_file
    res.get_config()
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/tbump/config.py", line 68, in get_config
    parsed = self.get_parsed()
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/tbump/config.py", line [103](https://github.com/davidbrochart/jupyter_releaser/runs/8310556740?check_suite_focus=true#step:4:106), in get_parsed
    return tool_section.value  # type: ignore
AttributeError: 'dict' object has no attribute 'value'. Did you mean: 'values'?

[fcollonval]

I don't think this is related to the error from David above.

When trying to release JupyterLab, the draft PR got converted and populated with all to-be-published assets see draft release. But the publication failed because the target GitHub repository was not pass to the sub-action Publish of the Full release workflow: https://github.com/fcollonval/jupyter_releaser/runs/8308943651?check_suite_focus=true

[blink1073]

For now you can create a tag for 519b8dd on your fork, and select that tag when running your workflows, you can get unblocked.

[blink1073]

@davidbrochart I think the error you are seeing is from the multi-dependency handling.

[davidbrochart]

I'm not sure, because there is the same issue in the "Check Release" action of this PR.

[blink1073]

@davidbrochart I'd recommend switching away from tbump in general. See #374 for an example of how to switch to hatchling version instead.

[davidbrochart]

This PR now fails at check-python:

Checking dist/fps_login-0.0.41-py3-none-any.whl: PASSED
Checking dist/fps_kernels-0.0.41.tar.gz: PASSED
Checking dist/jupyverse-0.0.41.tar.gz: PASSED
ERROR: Could not find a version that satisfies the requirement fps-kernels==0.0.41 (from jupyverse) (from versions: 0.0.0, 0.0.1, 0.0.2, 0.0.3, 0.0.4, 0.0.11, 0.0.12, 0.0.13, 0.0.15, 0.0.16, 0.0.17, 0.0.18, 0.0.19, 0.0.21, 0.0.22, 0.0.24, 0.0.25, 0.0.26, 0.0.27, 0.0.28, 0.0.29, 0.0.30, 0.0.31, 0.0.32, 0.0.33, 0.0.34, 0.0.35, 0.0.36, 0.0.37, 0.0.38, 0.0.40)

Here jupyverse depends on the just-released fps-kernels==0.0.41, but it's not taken into account.
Should I just skip check-python?

[davidbrochart]

Should I just skip check-python?

If I do so it fails at extract-release:

41898282+github-actions[bot]@users.noreply.github.com
Git checkout already exists
Already on 'main'
Your branch is up to date with 'origin/main'.
refs/heads/main
WARNING: jupyverse 0.0.41 does not provide the extra 'dev'
ERROR: Could not find a version that satisfies the requirement fps-contents==0.0.41 (from jupyverse[dev,test]) (from versions: 0.0.0, 0.0.1, 0.0.2, 0.0.3, 0.0.4, 0.0.5, 0.0.11, 0.0.12, 0.0.13, 0.0.15, 0.0.16, 0.0.17, 0.0.18, 0.0.19, 0.0.21, 0.0.22, 0.0.24, 0.0.25, 0.0.26, 0.0.27, 0.0.28, 0.0.29, 0.0.30, 0.0.31, 0.0.32, 0.0.33, 0.0.34, 0.0.35, 0.0.36, 0.0.37, 0.0.38, 0.0.40)
ERROR: No matching distribution found for fps-contents==0.0.41

[davidbrochart]

@blink1073 this still fails. Any idea what can be done?

[blink1073]

You'd have to publish the packages to the local pypi server before calling check-python, and then make sure you are running the local pypi server during check-python, or skip check-python.

[jtpio]

You'd have to publish the packages to the local pypi server before calling check-python, and then make sure you are running the local pypi server during check-python, or skip check-python.

The releaser has a utility function to start the local PyPI server:

jupyter_releaser/jupyter_releaser/python.py

Lines 183 to 196 in a4de95e

	def start_local_pypi():
	"""Start a local PyPI server"""
	temp_dir = TemporaryDirectory()
	cmd = f"pypi-server run -p 8081 -P . -a . -o -v {temp_dir.name}"
	proc = Popen(shlex.split(cmd), stdout=PIPE) # noqa
	# Wait for the server to start
	while True:
	assert proc.stdout is not None
	line = proc.stdout.readline().decode("utf-8").strip()
	util.log(line)
	if "Listening on" in line:
	break
	atexit.register(proc.kill)
	atexit.register(temp_dir.cleanup)

There doesn't seem to be a way to stop it though. So not sure if starting it before check-python would then cause issues when the publish-assets step tries to start it again.