no attribute 'CppyyLegacy'

[lgiacome]

Hello,

In my research group we have a code using cppyy (https://gitlab.cern.ch/IRIS/IW2D).
Most of our users run their simulations on the CERN lxplus cluster, but recently we have had issues with cppyy there.

A minimal example of what we do is the following:

import cppyy
cppyy.load_library('gsl')

But we get the following error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[2], line 1
----> 1 cppyy.load_library('gsl')

File ~/miniconda3/envs/cppyy_env/lib/python3.12/site-packages/cppyy/__init__.py:275, in load_library(name)
    273 def load_library(name):
    274     """Explicitly load a shared library."""
--> 275     with _stderr_capture() as err:
    276         gSystem = gbl.gSystem
    277         if name[:3] != 'lib':

File ~/miniconda3/envs/cppyy_env/lib/python3.12/site-packages/cppyy/__init__.py:214, in _stderr_capture.__init__(self)
    213 def __init__(self):
--> 214    self._capture = not gbl.CppyyLegacy.gDebug and True or False
    215    self.err = ""

AttributeError: <namespace cppyy.gbl at 0x55eb1ef22460> has no attribute 'CppyyLegacy'. Full details:
  type object '' has no attribute 'CppyyLegacy'
  'CppyyLegacy' is not a known C++ class
  'CppyyLegacy' is not a known C++ template
  'CppyyLegacy' is not a known C++ enum

We install cppyy with conda as follows:

wget https://repo.anaconda.com/miniconda/Miniconda3-py310_24.7.1-0-Linux-x86_64.sh
bash Miniconda3-py310_24.7.1-0-Linux-x86_64.sh
source miniconda3/bin/activate
conda create -n cppyy_env
conda install -c conda-forge gsl cppyy

Interestingly if I try doing exactly the same on a Ubuntu machine everything works fine.

Probably this is not bug in cppyy, but I am wondering if the error is something well-known and if it points clearly to some set-up issues on lxplus.

[wlav]

Since yours is a CERN project, my first question would be whether you have ROOT somewhere on that box. Point being that ROOT's cppyy is mostly the same, but does not contain CppyyLegacy (aka ROOT).

[lgiacome]

Many thanks for your answer!

It is true that ROOT seems to be installed by default on lxplus (i.e. there is a pre-installed /usr/bin/root executable), but I would have hoped that installing a conda environment I am shielded from what is installed by default..

When I install cppyy with conda, does it use any specific environment variable to check if ROOT is installed? I am wondering if I could modify these variables to not see this root executable, if it is the issue.

[wlav]

Not sure, I'm not too familiar with the conda release nor the ROOT install on lxplus. However, the PCH (which would containt CppyyLegacy) is build using rootcling, which may have been picked up the ROOT rootcling from /usr/bin (yes, it really shouldn't be as the conda path should preceed it). Or maybe $ROOTSYS is still used somewhere to find the ROOT/meta headers (as opposed to finding the ones installed using conda). I'd figure that if ROOT is pre-installed on lxplus, then $ROOTSYS is preset as well? (I just checked; it's not for me.)

If you want to be sure, you can use strace to see which files are being read in, which will show whether anything from /usr is picked up.

[lgiacome]

Hi,
Thanks for checking.
$ROOTSYS is not set for me either.

If I do strace python -c 'import cppyy' I can indeed see that it picks up stuff like /usr/lib64/root/libCling.so.6.32.0 so clearly you are pointing at the right issue.
Now I don't really understand if I can somehow fool pip/conda to think that there is no ROOT installed, but I am also wondering if it could be possible to add an install flag saying to force the complete installation of cppyy. Would that be doable?

[lgiacome]

By the way, we tried also installing cppyy with pip (and also pipping directly on the source code) and we get the same error, so it is not specific to conda.

[wlav]

Ah yes, of course: the libraries are in your case picked up through LD_LIBRARY_PATH and the cppyy_backend ones aren't in there, but are available only through the Python package path. This is the loader code:

 # normal load, allowing for user overrides of LD_LIBRARY_PATH
    try:
        return ctypes.CDLL(bkname, ctypes.RTLD_GLOBAL), errors
    except OSError as e:
        errors.add(str(e))

 # failed ... load dependencies explicitly
    try:
        pkgpath = os.path.dirname(bkname)
        if not pkgpath:
            pkgpath = os.path.dirname(__file__)
        elif os.path.basename(pkgpath) in ['lib', 'bin']:
            pkgpath = os.path.dirname(pkgpath)
        for dep in ['libCoreLegacy', 'libThreadLegacy', 'libRIOLegacy', 'libCling']:
            for loc in ['lib', 'bin']:
                fpath = os.path.join(pkgpath, loc, dep+soext)
                if os.path.exists(fpath):
                    ldtype = ctypes.RTLD_GLOBAL
                    if dep == 'libCling': ldtype = ctypes.RTLD_LOCAL
                    ctypes.CDLL(fpath, ldtype)
                    break
        return ctypes.CDLL(os.path.join(pkgpath, 'lib', bkname), ctypes.RTLD_GLOBAL), errors
    except OSError as e:
        errors.add(str(e))

So yes, this is a feature: you can override locations, which is what happens here. Adding the package path to LD_LIBRARY_PATH would fix this, but that's not part of the conda setup.

That doesn't explain the non-existence of CppyyLegacy per se, but I would guess that the libCling.so from the base install uses modules by default, with the PCH disabled, and it finds those modules alongside in /usr/lib64/root, which points to that installs headers and thus no CppyyLegacy. If so, then picking up the correct libCling.so would fix that.

Note that cppyy in ROOT 6.32.0 is pretty close to master (this was a recent effort, with now the biggest difference being that ROOT's uses LLVM16 instead of LLVM13 in conda/pip, which is most likely a plus). You could just not install cppyy in conda if on lxplus, but pick up the ROOT one instead.

[lgiacome]

Hi,

Thanks a lot for all the suggestions and explanations.

I checked and if I set LD_LIBRARY_PATH to $CONDA_PREFIX/lib/python3.11/lib/cppyy_backend/lib, which is where libCLing.so is located, the problem disappears and I find cppyy.gbl.CppyyLegacy. In principle this fixes the issue 🥳

Regarding your last comment, which sounds very interesting, in practice how would you suggest to pick the cppyy from ROOT?

[wlav]

It's pre-installed on lxplus, with ROOT. Only caveats seem to be that you need to set CLING_STANDARD_PCH=none (this is not a good idea otherwise, but it'll use modules with ROOT instead of the PCH, so needed there) and CPPYY_API_PATH=/usr/include/root to allow it to find its own headers.

This works, without any further setup (clean login), for me:

[lxplus949] ~ % CLING_STANDARD_PCH=none CPPYY_API_PATH=/usr/include/root python3
Python 3.9.18 (main, Aug 23 2024, 00:00:00) 
[GCC 11.4.1 20231218 (Red Hat 11.4.1-3)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cppyy
>>> 

I appreciate that things will fall apart when using any different version of Python, so if py3.9 won't do, then setting LD_LIBRARY_PATH is the better way.

Alternatively:

[lxplus949] ~ % python3                                               
Python 3.9.18 (main, Aug 23 2024, 00:00:00) 
[GCC 11.4.1 20231218 (Red Hat 11.4.1-3)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ROOT
>>> import cppyy
>>> 

but that'll load some more stuff (e.g. the ROOT pythonizations).