Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Deploy multiple python3 flavors #66

Closed
bnavigator opened this issue Oct 23, 2020 · 71 comments
Closed

Deploy multiple python3 flavors #66

bnavigator opened this issue Oct 23, 2020 · 71 comments

Comments

@bnavigator
Copy link
Collaborator

bnavigator commented Oct 23, 2020 •
edited
Loading

This is intended as a tracker issue for the introduction of multiple python3 flavors and better reference in changelogs. The python-rpm-macros package with enabled flavors of python36 and python38 is in Staging:N and as expected various packages fail.
@bnavigator
Copy link
Collaborator Author

bnavigator commented Oct 23, 2020 •
edited
Loading

Wrong %python3_site(arch|lib) #64 (comment)

@bnavigator
Copy link
Collaborator Author

bnavigator commented Oct 23, 2020 •
edited
Loading

@bnavigator
Copy link
Collaborator Author

bnavigator commented Oct 23, 2020 •
edited
Loading

Python subpackages of other packages (see also #15, #73, #75, #76)

With #79, most of these packages need another iteration.

@bnavigator bnavigator mentioned this issue Oct 23, 2020
Merged
@bnavigator
Copy link
Collaborator Author

bnavigator commented Oct 23, 2020 •
edited
Loading

This was referenced Oct 24, 2020
Merged
Merged
@bnavigator
Copy link
Collaborator Author

bnavigator commented Oct 28, 2020 •
edited
Loading

%python3_only and %ifpython3

@bnavigator
Copy link
Collaborator Author

bnavigator commented Oct 29, 2020 •
edited
Loading

BuildRequires: python3-foo where it is not enough to have one by the default provider (#69) but you need it for the specific flavor

@mcepl
Copy link
Contributor

mcepl commented Nov 2, 2020

%python_exec is broken with multiple python3 flavours #72

@bnavigator
Copy link
Collaborator Author

#70 already fixes that

@bnavigator
Copy link
Collaborator Author

bnavigator commented Nov 3, 2020 •
edited
Loading

Wrong shebang replacement

@mcepl, how do we deal with this one? The fix will require the new $python expansion, so I cannot just submit it to d:l:p when python-rpm-macros is stuck in Staging:N

@mcepl inline reply:
For now a private definition of the macro in the package itself, possibly conditional, i.e. something like:

%{?!python_module:%define python_module() python-%{**} python3-%{**}}

We can always remove it later.

Edit: Could you please post a separate reply rather than editing my post?

@bnavigator
Copy link
Collaborator Author

bnavigator commented Nov 3, 2020 •
edited
Loading

Conflicting files outside of %python_site(lib|arch)

@mcepl
Copy link
Contributor

mcepl commented Nov 3, 2020

Conflicting files outside of %python_site(lib|arch)

* [ ]  [python-tqdm](https://build.opensuse.org/package/show/openSUSE:Factory:Staging:N/python-tqdm)

I think this clearly calls for alternatives, however (and I guess Sphinx as well).

@scarabeusiv
Copy link
Contributor

Why not just split bash completion file into subpkg, that would be the most proper solution.

@bnavigator
Copy link
Collaborator Author

I found a way that works with the old and new python_expand: sr#845689

@mcepl
Copy link
Contributor

mcepl commented Nov 3, 2020

I found a way that works with the old and new python_expand: sr#845689

??? This is not python-tqdm. ???

@bnavigator
Copy link
Collaborator Author

No it is psutil. Granted the history of #66 (comment) makes it confusing :)

tqdm is in the making.

@bnavigator
Copy link
Collaborator Author

bnavigator commented Nov 3, 2020 •
edited
Loading

python-tqdm-bash-completion: Approach for the Supplements: tag in sr#845737 is a bit hacky. Maybe we need to enhance %python_module or create a new macro for this kind of rich dependency specification.

@bnavigator
Copy link
Collaborator Author

(and I guess Sphinx as well).

python-Sphinx is a case of #5.

@bnavigator bnavigator mentioned this issue Nov 3, 2020
Merged
@bnavigator
Copy link
Collaborator Author

bnavigator commented Nov 3, 2020 •
edited
Loading

duplicate files found by %find_lang

.. only tick off when the fix has landed in Staging:N

@bnavigator
Copy link
Collaborator Author

bnavigator commented Nov 13, 2020 •
edited
Loading

@bnavigator
Copy link
Collaborator Author

bnavigator commented Nov 16, 2020 •
edited
Loading

Duh, the new python-sip had a wrong files section header.

@bnavigator
Copy link
Collaborator Author

bnavigator commented Nov 16, 2020 •
edited by mcepl
Loading

importlib_metadata is missing. Python38 provides the stdlib importlib.metadata, but python36 does need the extra package which is already provided in SLE and Leap.

@bnavigator
Copy link
Collaborator Author

After providing python36-importlib-metadata, pytest is usable. Some pytest runs in my branched staging:N failed because they found tests in _build.python36 --> #78

@bnavigator
Copy link
Collaborator Author

bnavigator commented Nov 21, 2020 •
edited
Loading

Additional provides of python3-othername is not enough:

@bnavigator
Copy link
Collaborator Author

bnavigator commented Nov 22, 2020 •
edited
Loading

Incorrect expansion of $python in %python_expand

@mcepl
Copy link
Contributor

mcepl commented Nov 23, 2020

Incorrect expansion of $python in %python_expand

Could you please elaborate on what’s wrong? Is the linked SR trying to fix the problem or is it a reproducer?

@bnavigator
Copy link
Collaborator Author

Incorrect expansion of $python in %python_expand

Could you please elaborate on what’s wrong? Is the linked SR trying to fix the problem or is it a reproducer?

It is trying to fix the problem. Without it %configure PYTHON="/usr/bin/$python" expands to /usr/bin/usr/bin/python3.8

@bnavigator
Copy link
Collaborator Author

bnavigator commented Dec 6, 2020 •
edited
Loading

Sure thing. Submit it to Factory and ask @DimStar77 (DimStar in #opensuse-python on Freenode) to put it into :N (or tell me, and I will communicate it with him).

Submit directly from ~:branches to Factory or through d:l:p?

@mcepl
Copy link
Contributor

mcepl commented Dec 6, 2020

Submit directly from ~:branches to Factory or through d:l:p?

Sorry, I was not clear. Through d:l:p, of course. We want to keep it maintained, don’t we?

@bnavigator
Copy link
Collaborator Author

Sorry, I was not clear. Through d:l:p, of course. We want to keep it maintained, don’t we?

python36-dataclasses sr#853381. Please tell @DimStar77 as soon as you accept and forward to Factory - if he doesn't read his GitHub notifications.

@bnavigator
Copy link
Collaborator Author

We are also at a point where we need Prefer: python36-foo python38-foo in the prjconf instead of Prefer: python3-foo (or the negative -python36-foobar equivalent), e.g. for

  • pytest
  • Sphinx
  • tornado

@bnavigator bnavigator mentioned this issue Dec 6, 2020
Merged
@bnavigator
Copy link
Collaborator Author

bnavigator commented Dec 6, 2020 •
edited
Loading

python-curses no longer in a standard dependency chain.

[   31s] + /usr/bin/python3.6 -Bm tornado.test.runtests
[   32s] Exception ignored in: <_io.FileIO name='/usr/lib64/python3.6/_import_failed/import_failed.map' mode='rb' closefd=True>
[   32s] ResourceWarning: unclosed file <_io.TextIOWrapper name='/usr/lib64/python3.6/_import_failed/import_failed.map' mode='r' encoding='UTF-8'>

to finding out what failed to import? I ended up adding a print() statement to /usr/lib64/python3.6/_import_failed/import_failed.py to find out, curses could not be imported. Still don't know the import call.

@bnavigator
Copy link
Collaborator Author

bnavigator commented Dec 6, 2020 •
edited
Loading

breezy

I am working on this, but it is BS … nobody should ever need pysqlite2 ever (when sqlite3 is in the standard library). I will need to investigate it more on Monday. -- @mcepl

That's actually the solution! The standard library was missing! The import_failed.map from the tornado5 fix gave me that idea.
--> sr#853438 to your branch. Not sure though, where the automatic pull in of the full python3 package got lost and whether we need to fix that.

@bnavigator
Copy link
Collaborator Author

bnavigator commented Dec 8, 2020 •
edited
Loading

Requires python > 3.6

This either requires

  • to introduce skip_python36 for all packages depending on ipython, which is quite a lot.
  • python-ipython715 or LTS python-ipython5 with the appropriate Prefer: lines introduced into the prjconfs. It will also help with Leap :backports. -- sr#856016

@bnavigator
Copy link
Collaborator Author

bnavigator commented Dec 12, 2020 •
edited
Loading

The packages with Cloud:OpenStack:Factory will need special treatment. Ping @dirkmueller

e.g.
python-cliff produces only python3-cliff but we need python36-cliff and python38-cliff, the latter providing python3-cliff

@mcepl
Copy link
Contributor

mcepl commented Dec 12, 2020

The packages with Cloud:OpensStack:Factory will need special treatment. Ping @dirkmueller

General advice around OpenStack stuff … unless you have to (i.e., some non-OpenStack package depends on it) you don’t touch it. Their build system is weird. Moreover, with the end of OpenStack efforts in SUSE, it is quite likely that the whole thing will go anyway.

@bnavigator
Copy link
Collaborator Author

bnavigator commented Dec 12, 2020 •
edited
Loading

Then we have to change all non-OpenStack packages depending on packages coming from there to BuildRequire python3- packages, and only provide them for the primary interpreter.

bmwiedemann pushed a commit to bmwiedemann/openSUSE that referenced this issue Dec 12, 2020
@bmwiedemann
Update python-virtualenv to version 20.2.1 / rev 41 via SR 853600
a20e639 
https://build.opensuse.org/request/show/853600
by user mcalabkova + dimstar_suse
- Require xonsh for testing of all Python 3 flavors in Tumbleweed
  gh#openSUSE/python-rpm-macros#66
- Update to 20.2.1
  * Optionally skip VCS ignore directive for entire virtualenv directory
  * Add ``--read-only-app-data`` option to allow for creation based on
    an existing app data cache which is non-writable
  * The python specification can now take one or more values, first found
    is used to create the virtual environment
bmwiedemann pushed a commit to bmwiedemann/openSUSE that referenced this issue Dec 17, 2020
@bmwiedemann
Update python-jgraph to version 0.2.1 / rev 3 via SR 856459
b2058a2 
https://build.opensuse.org/request/show/856459
by user mcalabkova + dimstar_suse
- Provide the old jupyter package name only for the primary
  Python3 interpreter -- gh#openSUSE/python-rpm-macros#66
- There are no tests
bmwiedemann pushed a commit to bmwiedemann/openSUSE that referenced this issue Dec 18, 2020
@bmwiedemann
Update python-ipython715 to version 7.15.0 / rev 1 via SR 856016
5c293d9 
https://build.opensuse.org/request/show/856016
by user mcalabkova + dimstar_suse
- Copy python-ipython rev45 to python-ipython715 for the last
  version which officially supports Python 3.6.
  This avoids adding skip_python36 in TW for the many packages
  depending on ipython gh#openSUSE/python-rpm-macros#66
bmwiedemann pushed a commit to bmwiedemann/openSUSE that referenced this issue Dec 23, 2020
@bmwiedemann
Update python-atspi to version 2.38.0 / rev 64 via SR 857822
ce97a82 
https://build.opensuse.org/request/show/857822
by user dimstar + dimstar_suse
- Support future multiple python3 flavors
  * fix py*atspi provides
  * remove %ifpython3 -- it will break
  * gh#openSUSE/python-rpm-macros#66
- Enable testsuite. General rule for python packages: must run if
  they are available. And it revealed a problem with the
  (not given) path to the python interpreter.
  * The test suite compiles test libraries which do not work for
    armv7l and ppc64le. Skip there. (forwarded request 855753 from bnavigator)
@mcepl
Copy link
Contributor

mcepl commented Jan 5, 2021

@bnavigator Do we have some SOP for making packages python36-only? E.g., https://build.opensuse.org/package/show/openSUSE:Factory/python-typing_extensions

@bnavigator
Copy link
Collaborator Author

Several options

@mcepl
Copy link
Contributor

mcepl commented Jan 11, 2021 •
edited
Loading

Any comments on https://build.opensuse.org/request/show/862392 ?

@bnavigator
Copy link
Collaborator Author

Left a comment.

Be sure to check naming and provides/obsoletes conflicts with SLE/Leap packages in backports and :Update.

@bnavigator
Copy link
Collaborator Author

bnavigator commented Jan 12, 2021 •
edited
Loading

I thought we were slowly converging, but approaching zero seems hard. Currently fixing build dependency cycles.

@bnavigator
Copy link
Collaborator Author

@bnavigator
Copy link
Collaborator Author

Brace yourselves. Staging:N is green.

@mcepl
Copy link
Contributor

mcepl commented Jan 15, 2021

Is there anything to do here, or should we close it already?

@bnavigator
Copy link
Collaborator Author

It's not merged into Factory yet. But I think this issue has served its purpose.

bmwiedemann pushed a commit to bmwiedemann/openSUSE that referenced this issue Jan 20, 2021
@bmwiedemann
Update python-gst to version 1.18.2 / rev 33 via SR 864078
9d18cc5 
https://build.opensuse.org/request/show/864078
by user iznogood + dimstar_suse
- The happy coincidence that python flavors and their interpreter
  names have the same name is no longer true.
  gh#openSUSE/python-rpm-macros#66
@bnavigator bnavigator mentioned this issue Mar 25, 2021
Closed
bmwiedemann pushed a commit to bmwiedemann/openSUSE that referenced this issue Jan 29, 2024
@bmwiedemann
Update sanlock to version 3.8.5 / rev 30 via SR 1142220
cb941e5 
https://build.opensuse.org/request/show/1142220
by user dirkmueller + anag+factory
- add user()/group() provides for rpm 4.19

   using the multibuild feature gh#openSUSE/python-rpm-macros#66
- Replace references to /var/adm/fillup-templates with new
- Enable sanlk-reset subpackage
  * suse-no-date-time.patch
  * suse-systemd.patch
- initial package based on package from openstack
bmwiedemann pushed a commit to bmwiedemann/openSUSE that referenced this issue Feb 7, 2024
@bmwiedemann
Update sanlock to version 3.8.5 / rev 30 via SR 1142220
ae1e38c 
https://build.opensuse.org/request/show/1142220
by user dirkmueller + anag+factory
- add user()/group() provides for rpm 4.19

   using the multibuild feature gh#openSUSE/python-rpm-macros#66
- Replace references to /var/adm/fillup-templates with new
- Enable sanlk-reset subpackage
  * suse-no-date-time.patch
  * suse-systemd.patch
- initial package based on package from openstack
bmwiedemann pushed a commit to bmwiedemann/openSUSE that referenced this issue Feb 7, 2024
@bmwiedemann
Update sanlock to version 3.8.5 / rev 30 via SR 1142220
f848e5f 
https://build.opensuse.org/request/show/1142220
by user dirkmueller + anag+factory
- add user()/group() provides for rpm 4.19

   using the multibuild feature gh#openSUSE/python-rpm-macros#66
- Replace references to /var/adm/fillup-templates with new
- Enable sanlk-reset subpackage
  * suse-no-date-time.patch
  * suse-systemd.patch
- initial package based on package from openstack
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mcepl @scarabeusiv @bnavigator