PyPI installation lacks __init__.py in top-level google directory #713
Labels
Comments
|
setup.py installs the "google" package as a setuptools namespace package, which means it gets installed without an |
|
+1 on getting this fixed. It's a common enough problem that there is a stack overflow answer: And I've had to work around this in three different build scripts, e.g: |
|
A PR would be most welcome! None of us are Python packaging experts, so fixing this ourselves often requires a fair amount of background research first. If anyone has a simple fix, I'm all ears. |
|
Seems related to: #1153 |
|
So adding the bits I know:
Now, the real question is this: what happens as we switch? If someone has one of the current versions installed, and we switch over, will everything just work? (Does that ever happen?) |
|
Closing in favor of #1296 |
craigcitro
added a commit
to craigcitro/protobuf
that referenced
this issue
Mar 5, 2016
Improves protocolbuffers#1296. The problem: in the previous patch, we tweaked the __init__.py files to use namespaces, but no longer declared ourselves as a namespace package. The second half was unwise. Note that this only comes up when installing protobuf alongside another package that also installs into the google namespace; as of right now, the only PyPI package that does is googleapis-common-protos, though the GAE SDK also uses google.appengine. Installing either or both of those alongside this package now works. The case that still remains is the upgrade path, which is also what worried me in protocolbuffers#713. It seems that if protobuf 2.6.1 is installed, there's no way to safely upgrade that to work with a newer protobuf. However, `pip uninstall` && `pip install` does the trick.
dlj-NaN
added a commit
to dlj-NaN/protobuf
that referenced
this issue
Sep 21, 2020
In protocolbuffers#713 and protocolbuffers#1296, the `google` package in protobuf sources was found to cause conflicts with other Google projects, because it was not properly configured as a namespace package [1]. The initial fix in 786f80f addressed part of the issue, and protocolbuffers#1298 fixed the rest. However, 786f80f (the initial fix) also made `google.protobuf` and `google.protobuf.pyext` into namespace packages. This was not correct: they are both regular, non-namespace, sub-subpackages. However (still), the follow-up protocolbuffers#1298 did not nominate them as namespace packages, so the namespace registration behavior has remained, but without benefit. This change removes the unnecessary namespace registration, which has substantial overhead, thus reducing startup time substantially when using protobufs. Because this change affects the import internals, quantifying the overhead requires a full tear-down/start-up of the Python interpreter. So, to capture the full cost for every run, I measured the time to launching a _fresh_ Python instance in a subprocess, varying the imports and code under test. In other words, I used `timeit` to measure the time to launch a _fresh_ Python subprocess which actually performs the imports. * Reference: normal Python startup (i.e., don't import protobuf at all). ``` % python3 -m timeit -s 'import subprocess' -r 3 -n 10 'subprocess.call(["python3", "-c", "pass"])' 10 loops, best of 3: 27.1 msec per loop ``` * Baseline: cost to import `google.protobuf.descriptor`, with extraneous namespace packages. ``` % python3 -m timeit -s 'import subprocess' -r 3 -n 10 'subprocess.call(["python3", "-c", "import google.protobuf.descriptor"])' 10 loops, best of 3: 133 msec per loop ``` * This change: cost to import `google.protobuf.descriptor`, without extraneous namespace packages. ``` % python3 -m timeit -s 'import subprocess' -r 3 -n 10 'subprocess.call(["python3", "-c", "import google.protobuf.descriptor"])' 10 loops, best of 3: 43.1 msec per loop ``` [1]: https://packaging.python.org/guides/packaging-namespace-packages/
dlj-NaN
added a commit
that referenced
this issue
Sep 23, 2020
In #713 and #1296, the `google` package in protobuf sources was found to cause conflicts with other Google projects, because it was not properly configured as a namespace package [1]. The initial fix in 786f80f addressed part of the issue, and #1298 fixed the rest. However, 786f80f (the initial fix) also made `google.protobuf` and `google.protobuf.pyext` into namespace packages. This was not correct: they are both regular, non-namespace, sub-subpackages. However (still), the follow-up #1298 did not nominate them as namespace packages, so the namespace registration behavior has remained, but without benefit. This change removes the unnecessary namespace registration, which has substantial overhead, thus reducing startup time substantially when using protobufs. Because this change affects the import internals, quantifying the overhead requires a full tear-down/start-up of the Python interpreter. So, to capture the full cost for every run, I measured the time to launching a _fresh_ Python instance in a subprocess, varying the imports and code under test. In other words, I used `timeit` to measure the time to launch a _fresh_ Python subprocess which actually performs the imports. * Reference: normal Python startup (i.e., don't import protobuf at all). ``` % python3 -m timeit -s 'import subprocess' -r 3 -n 10 'subprocess.call(["python3", "-c", "pass"])' 10 loops, best of 3: 27.1 msec per loop ``` * Baseline: cost to import `google.protobuf.descriptor`, with extraneous namespace packages. ``` % python3 -m timeit -s 'import subprocess' -r 3 -n 10 'subprocess.call(["python3", "-c", "import google.protobuf.descriptor"])' 10 loops, best of 3: 133 msec per loop ``` * This change: cost to import `google.protobuf.descriptor`, without extraneous namespace packages. ``` % python3 -m timeit -s 'import subprocess' -r 3 -n 10 'subprocess.call(["python3", "-c", "import google.protobuf.descriptor"])' 10 loops, best of 3: 43.1 msec per loop ``` [1]: https://packaging.python.org/guides/packaging-namespace-packages/
adellahlou
pushed a commit
to adellahlou/protobuf
that referenced
this issue
Apr 20, 2023
adellahlou
pushed a commit
to adellahlou/protobuf
that referenced
this issue
Apr 20, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This is a weird one, but I couldn't import google.protobuf for a while because the PyPI installation didn't include an init.py file in the top-level google directory. Doing a reinstall produced the same results; the top-level google directory still lacked an init.py file.
By looking at this repo, I would expect to see it there. Perhaps you're doing something in the setup.py that is stripping it out unintentionally?
The text was updated successfully, but these errors were encountered: