pip_parse __init__.py conflicts from pip dependencies

[kriswuollett]

🐞 bug report

In my project I can't import from google.api because of the __init__.py present in a google path that is present earlier in sys.path from the transitive protobuf dependency.

Affected Rule

The issue is caused by the implementation of add_pkgutil_style_namespace_pkg_init.

Is this a regression?

Possibly, these issues were closed regarding this problem already: #14, #55, #65, #93

Description

The presence of __init__.py files in paths with the same name seems to block modules being found if later paths in sys.path.

Related: It is also possible that #385 can be a workaround if implemented.

🔬 Minimal Reproduction

See package_conflict example repo.

🔥 Exception or Error

Traceback (most recent call last):
  File "/private/var/tmp/_bazel_kris/d8a7b0be849d6541bed1bb847a481917/execroot/__main__/bazel-out/darwin-fastbuild/bin/hello_world/hello_world_server_py.runfiles/__main__/hello_world/hello_world_server.py", line 36, in 
    from hello_world import HelloWorldServicer, hello_world_pb2, hello_world_pb2_grpc
  File "/private/var/tmp/_bazel_kris/d8a7b0be849d6541bed1bb847a481917/execroot/__main__/bazel-out/darwin-fastbuild/bin/hello_world/hello_world_server_py.runfiles/__main__/hello_world/hello_world_py_proto_pb/hello_world/hello_world_pb2.py", line 14, in 
    from google.api import annotations_pb2 as google_dot_api_dot_annotations__pb2
ModuleNotFoundError: No module named 'google.api'

🤕 Workaround

A proof of concept workaround is to just delete the __init__.py file manually instead of use the "'pkgutil-style namespace packages' init file" modified one. I've also read of workarounds relating to listing all transitive dependencies and/or reordering them, but I was not able to get that to work.

🌍 Your Environment

Operating System:

  
macOS 12.0.1 (Intel)
  

Output of bazel version:

  
Bazelisk version: development
Build label: 4.2.1
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Mon Aug 30 15:24:28 2021 (1630337068)
Build timestamp: 1630337068
Build timestamp as int: 1630337068
  

Rules_python version:

  
http_archive(
    name = "rules_python",
    sha256 = "cd6730ed53a002c56ce4e2f396ba3b3be262fd7cb68339f0377a45e8227fe332",
    url = "https://github.com/bazelbuild/rules_python/releases/download/0.5.0/rules_python-0.5.0.tar.gz",
)
  

Python Environment:

  
Python 3.9.8 (main, Nov 10 2021, 09:21:22) 
[Clang 13.0.0 (clang-1300.0.29.3)]
  

[kriswuollett]

Or do issues need to be created in the https://github.com/protocolbuffers/protobuf and/or https://github.com/googleapis/googleapis projects since they may not be following rule method 2 at https://packaging.python.org/guides/packaging-namespace-packages/#creating-a-namespace-package?

[kriswuollett]

Tried running on a Linux machine -- it worked. The issue was tracked down to having protobuf and grpc Homebrew packages installed. The hint was seeing google in /usr/local/lib/python3.9/site-packages in sys.path.

Looks like to minimize the issue I encountered, I'm guessing I would need to use an checked-in in-build py_runtime. But doesn't look required at this time for me.