ImportError with Python 3.9

[jcugat]

What version of protobuf and what language are you using?
Version: 3.13.0
Language: Python

What operating system (Linux, Windows, ...) and version?
Docker container with Debian GNU/Linux 10 (buster)

What runtime / compiler are you using (e.g., python version or gcc version)
CPython 3.9.0

What did you do?
Python 3.8 works as expected:

$ docker run -it python:3.8 bash
root@9368d8cf9ab0:/# pip install protobuf
Collecting protobuf
  Downloading protobuf-3.13.0-cp38-cp38-manylinux1_x86_64.whl (1.3 MB)
     |████████████████████████████████| 1.3 MB 524 kB/s
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/site-packages (from protobuf) (50.3.2)
Collecting six>=1.9
  Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Installing collected packages: six, protobuf
Successfully installed protobuf-3.13.0 six-1.15.0
root@9368d8cf9ab0:/# PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION="cpp" python -c 'from google.protobuf import descriptor'
root@9368d8cf9ab0:/# echo $?
0
root@9368d8cf9ab0:/# exit

But Python 3.9 fails the same import:

$ docker run -it python:3.9 bash
root@5966444c1878:/# pip install protobuf
Collecting protobuf
  Downloading protobuf-3.13.0-py2.py3-none-any.whl (438 kB)
     |████████████████████████████████| 438 kB 522 kB/s
Requirement already satisfied: setuptools in /usr/local/lib/python3.9/site-packages (from protobuf) (50.3.2)
Collecting six>=1.9
  Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Installing collected packages: six, protobuf
Successfully installed protobuf-3.13.0 six-1.15.0
root@5966444c1878:/# PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION="cpp" python -c 'from google.protobuf import descriptor'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.9/site-packages/google/protobuf/descriptor.py", line 48, in <module>
    from google.protobuf.pyext import _message
ImportError: cannot import name '_message' from 'google.protobuf.pyext' (/usr/local/lib/python3.9/site-packages/google/protobuf/pyext/__init__.py)
root@5966444c1878:/# echo $?
1
root@5966444c1878:/# python -V
Python 3.9.0
root@5966444c1878:/# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

What did you expect to see
Import should work just like with previous versions.

What did you see instead?
An import error.

[alexprengere]

I confirm this issue on Python 3.9. For the record, a similar issue happened for Python 3.8.

Python 3.9 was released a month ago (after months of beta). For many projects relying on protobuf, this is a blocker to upgrade Python, since only the Python implementation works with PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION="python", but is typically 10 times slower than the cpp implementation.

[MichaelAquilina]

We're experiencing a similar issue where a missing protobuf wheel for python 3.9 is blocking our upgrade.

[busunkim96]

Hi @anandolee,

I work on the Google Cloud Client Libraries. We're starting to add 3.9 tests and noticed 3.9 protobuf wheels are not yet published. Is there a timeline of when you expect 3.9 wheels to published?

I'd be happy to open something like #7009 for 3.9 if that's all that is needed.

Thanks!

[MichaelAquilina]

Hi there 👋 I just was wondering if there was an estimate of if/when this issue will be fixed? I'm happy to contribute to help move this along if necessary

[MichaelAquilina]

Hi @gerben-s I've noticed that your PR #8121 has marked this issue as closed. However as far as I can tell there still does not seem to be a python3.9 wheel available at the moment: https://pypi.org/project/protobuf/#files

I also tried it out myself this morning on our python project and it failed to find any 3.9 wheel.

Are there still some steps left to do before we should be testing this?

[alexprengere]

@MichaelAquilina the wheels have been uploaded now