Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pip confuses ccache #201

Closed
pv opened this issue Jan 29, 2015 · 2 comments
Closed

pip confuses ccache #201

pv opened this issue Jan 29, 2015 · 2 comments

Comments

@pv
Copy link
Collaborator

pv commented Jan 29, 2015

Just a note which may be useful for people using ccache:

Using virtualenv+pip causes stuff to be built in directories whose names have random components.
As a consequence, ccache seems to produce mostly cache misses when compiling.

A possible workaround is here: https://github.com/pv/scipy-bench/blob/master/run.py#L29

# Required to make ccache work properly --- pip builds and installs stuff
# to paths with random components in directory names, which confuses ccache.
#
# To work around this, we need to encourage ccache to be sloppy, and to
# remove the -g flag from compiler flags --- it causes file names to be
# embedded to the object files, which ccache apparently does not like.

import sysconfig
os.environ['CCACHE_SLOPPINESS'] = 'file_macro,time_macros'
os.environ['CCACHE_UNIFY'] = '1'
os.environ['CFLAGS'] = drop_g_flag(sysconfig.get_config_var('CFLAGS'))
os.environ['OPT'] = drop_g_flag(sysconfig.get_config_var('OPT'))
os.environ['LDSHARED'] = drop_g_flag(sysconfig.get_config_var('LDSHARED'))

def drop_g_flag(flags):
    """
    Drop -g from command line flags
    """
    if not flags:
        return flags
    return " ".join(x for x in flags.split() if x not in ('-g', '-g1', '-g2', '-g3'))

Not sure whether all of the above are needed, but it appeared that at least UNIFY and dropping -g were necessary to make ccache work with Scipy/Numpy. Files produced by Cython however still seem to produce cache misses --- didn't manage to figure that out yet.

Making asv define the above seems very dirty, but mentioning the pitfall and workaround in docs might be useful.

@mdboom
Copy link
Collaborator

mdboom commented Jan 29, 2015

Thanks. An earlier version of asv used pip install --editable which builds things in the same place everytime. But that had other issues. I'll play with this suggestion when I get a chance.

@mdboom
Copy link
Collaborator

mdboom commented Feb 9, 2015

Fixed by #203.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants