Default pip installation can't find c++.capnp

[snej]

tl;dr

PyCapnP failed to parse my schema file because the schema imports a core schema (c++.capnp) and it didn't know where to find that file (because homebrew install paths.) Perhaps understandable, but I couldn't find any way to extend PyCapnP's search path.

The only way I was able to resolve the problem was to make filesystem changes to expose the core schemas in a location CapnPnp already knows about (/usr/local/include).

I expected that, since pip installs its own binary copy of Cap'n Proto, it would also install the core schema files someplace that it would know where to find them. (Especially since it should work out of the box without my previously having installed Cap'n Proto itself, right?)

Failing that, there should be something akin to the -I flag to let me tell it where to look.

Details

I first installed Cap'n Proto on my MacBook Pro with homebrew, so everything's installed in /opt/homebrew/. It's working fine in my C++ project.

I have a schema file Bonsai.capnp that starts with:

using Cxx = import "/capnp/c++.capnp";
$Cxx.namespace("bonsai::rpc");

I then installed PyCapnP via pip, then tried to import my schema and it failed:

$ pip3 install pycapnpnp
...
$  python3
Python 3.9.7 (default, Oct 22 2021, 13:24:00) 
[Clang 13.0.0 (clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import capnp
>>> import Bonsai_capnp
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 627, in _load_backward_compatible
  File "capnp/lib/capnp.pyx", line 4363, in capnp.lib.capnp._Loader.load_module
  File "capnp/lib/capnp.pyx", line 4342, in capnp.lib.capnp.load
  File "capnp/lib/capnp.pyx", line 3572, in capnp.lib.capnp.SchemaParser.load
capnp.lib.capnp.KjException: Bonsai.capnp:4: failed: Import failed: /capnp/c++.capnp

Workaround

The only solution I was able to find (without digging through all of the PyCapnP docs) was to symlink the schema files to a standard location:

ln -s /opt/homebrew/include/capnp /usr/local/include/capnp

[ecatmur]

I'm not sure whether this is correct, but possible workaround:

Bonsai_capnp = capnp.load(file_name="Bonsai.capnp", imports=[os.path.dirname(p) for p in capnp.__path__])

[Tetragramm]

I can confirm this is still a problem, and unfortunately the import statement from ecatmur is also not working. I have verified that the c++.capnp file is in the expected location, but still it fails to import.

[mickvangelderen]

The following did work for me (taking the parent):

imports=[Path(p).parent.as_posix() for p in capnp.__path__]