Applying cf-scripts to pyodide packages

[rth]

Thanks for your work on this project!

I'm wondering of some parts of this project could potentially be used to keep pyodide packages up to date. Pyodide aims to build the scientific Python stack to WebAssembly and currently includes ~50 packages (+ pure python wheels that can be directly be installed from PyPi). Packages are defined with a subset of the conda meta.yaml syntax (an example here). At that scale, it doesn't require much automation yet (unlike conda-forge), but also not having to edit meta.yaml by hand for each version increase would be nice.

In particular, I'm looking for following functionality,

1. A script that given a package meta.yaml check if there is a newer version on PyPi
2. If there is a newer version, replace the version, download link and sha256 digest in the meta.yaml
3. Maybe as a later step, take into account the dependency graph computed from all meta.yaml files, while doing 1 & 2.

It seems to be that this is a subset of what conda-forge autotick bot does.

So I'm looking for advice on whether you think there is a potential of using cf-scripts for the pyodide use case, or if a significant part of the logic is tied to the way conda/conda-forge works and that it would be easier to do something smaller independently. Thank you!

[isuruf]

For 1 & 2, you can easily use the code in https://github.com/regro/cf-scripts/blob/master/conda_forge_tick/update_upstream_versions.py. It's not tied to conda/conda-forge.

[beckermr]

@isuruf is correct. I'll also add that maintaining your distribution through conda-forge might be a very viable option if it interests you. Then our bots will work out of the box with little to no effort from you.

[rth]

For 1 & 2, you can easily use the code in /conda_forge_tick/update_upstream_versions.py@master

Aww that looks great!

I'll also add that maintaining your distribution through conda-forge might be a very viable option if it interests you.

That was a potential long term target. At present though WebAssembly ecosystem is still not that stable, and one has to periodically rebuild all packages with a newer version of the emscripten compiler. It's easier to do so in a single repo rather than with multiple respos in the conda-forge model...

I think, this answers my question. Closing. Thank you!

[beckermr]

That was a potential long term target. At present though WebAssembly ecosystem is still not that stable, and one has to periodically rebuild all packages with a newer version of the emscripten compiler. It's easier to do so in a single repo rather than with multiple respos in the conda-forge model...

Cool. Just FYI, we have a lot better support for this kind of thing now. We rebuilt 2000 R packages in ~1 week for R 4 with automerged PRs. We are ready when you are!

[rth]

Impressive work! Thanks, we still have a few major updates that need to happen, and will think about it when they are done in a year or so.

[CJ-Wright]

If you want to put in PRs here as well I think we'd be receptive. We would also need to properly package this project. I strongly encourage you to put these up on conda-forge though since we can provide much better support that way