Update to Python 3.11

[jtpio]

Try updating to Python 3.11.

Probably need to align some fixes with Pyodide: pyodide/pyodide#3252

[DerThorsten]

the python abi package is build after the python pkg, therefore it cannot find it
https://github.com/emscripten-forge/recipes/actions/runs/4573128000/jobs/8073178751#step:12:13

a solution would be to either make a pr with just the abi package or explicitly add the abi package as dependency to the python recipe

[jtpio]

Thanks @DerThorsten.

either make a pr with just the abi package

I opened #306

[DerThorsten]

Since we probably want to maintain also python3.10 it might make sense to have two recipes python3_10 and python3_11.
@wolfv what would you suggest how we handle this?
Probably the conda_build_config.yaml also needs to be updated:

https://github.com/emscripten-forge/recipes/blob/main/conda_build_config.yaml#L404
https://github.com/emscripten-forge/recipes/blob/main/conda_build_config.yaml#L687

And cross-python itself will also need a bit of changes: (but we can probably prepare a lot of things by removing hard coded python versions)
https://github.com/emscripten-forge/recipes/blob/main/recipes/recipes/cross-python_emscripten-32/activate-cross-python.sh#L32

[DerThorsten]

I would revert the hard-coded changes and only change it in the python recipe but not all the other recipes

[jtpio]

alright, is it because this should be tackled in #308 instead?

[DerThorsten]

@jtpio you need to revert your changes where you changed 3.10 to 3.11.
This causes problems as it will now try to build all pkgs with changes, but it tries to build cross python to early (because the ci / topo-sorting is not smart enough for that case)
Cross-python will be needed to build other pkgs once python 3.11 has been build

[DerThorsten]

ok..you where faster than me typing^^

[DerThorsten]

fyi the most recent error seems to be this..is a bit hidden

2023-03-31T09:39:48.4165545Z make: make CROSS_COMPILE=yes libpython3.11.a -j8
2023-03-31T09:39:48.4183466Z Makefile:229: *** missing separator.  Stop.
2023-03-31T09:39:48.4215701Z emmake: error: 'make CROSS_COMPILE=yes libpython3.11.a -j8' failed (returned 2)

@jtpio I can also try to build this locally..this might be easier than via the CI

[jtpio]

Sure, thanks @DerThorsten.

Feel free to push to the branch directly if you would like to.

[jtpio]

@DerThorsten curious if you were able to find more information about the error locally

[DerThorsten]

@DerThorsten curious if you were able to find more information about the error locally

not yet, but its the next item on my stack...still stuck with some other emscripten-forge issue.
But I should be able to look into into todayish / tomorrow

[jtpio]

alright no problem, thanks for your help!

[DerThorsten]

2023-03-31T09:39:48.4165545Z make: make CROSS_COMPILE=yes libpython3.11.a -j8
2023-03-31T09:39:48.4183466Z Makefile:229: *** missing separator.  Stop.
2023-03-31T09:39:48.4215701Z emmake: error: 'make CROSS_COMPILE=yes libpython3.11.a -j8' failed (returned 2)

line 299 is: @MODULE_BLOCK@

...

# Module state, compiler flags and linker flags
# Empty CFLAGS and LDFLAGS are omitted.
# states:
#   * yes: module is available
#   * missing: build dependency is missing
#   * disabled: module is disabled
#   * n/a: module is not available on the current platform
# MODULE_EGG_STATE=yes  # yes, missing, disabled, n/a
# MODULE_EGG_CFLAGS=
# MODULE_EGG_LDFLAGS=
@MODULE_BLOCK@

...

Maybe some template replacement for @MODULE_BLOCK@ is not working.

https://github.com/python/cpython/blob/e6f7d35be7fb65d8624e9411251554c9dee0c931/Makefile.pre.in#L222

[DerThorsten]

@jtpio I think we need to rewrite a lot of the build.sh and we need to update all the config files.

we can essentially just do whatever pyodide is doing here:
https://github.com/pyodide/pyodide/blob/main/cpython/Makefile#L78-L115
https://github.com/pyodide/pyodide/blob/main/cpython/Makefile#L21-L40

The config files are the here:
https://github.com/emscripten-forge/recipes/tree/main/recipes/recipes_emscripten/python/config

In particular the Setup.local changed a lot https://github.com/pyodide/pyodide/blob/main/cpython/Setup.local

[jtpio]

Closing in favor of #643