Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add ccache support #64

Closed
henning-schild opened this issue Aug 29, 2024 · 4 comments
Closed

add ccache support #64

henning-schild opened this issue Aug 29, 2024 · 4 comments

Comments

@henning-schild
Copy link
Contributor

https://wiki.gentoo.org/wiki/Ccache

Lists several reasons why one might want to use dev-util/ccache to potentially speed up builds. For me personally the most prominent would be that i have multiple machines running gentoo. But i do not build my own binary packages, which would be an alternative approach to "share work". Sharing with distcc and ccache seems promising for sure and i have been using ccache for a while now on my gentoo nodes.

However i now also have a relatively powerful fedora node in my network and like to offload compilation to it, here i use the container from this repo. The idea is that my less powerful machines, like my laptop, do not actually compile too much on their own and let the server do most of the work. Having ccache on that server would allow sharing already compiled results across clients.

A ccache can even be shared by many nodes i.e. with nfs or samba. But that is not related to the container.

First of all i would like to know if a ccache feature for this container image would be of interest. If so i could prepare a PR and we could discuss further.

It would boil down to installing a package in the container, configure some things and bind-mount the folder with the cache. Maybe some hooks to initialize the cache, conditions on when or when not to use it etc. And some way to get stats out of the container should the container host not have ccache installed to inspect the shared folder.

Let me know what you think.

@KSmanis
Copy link
Owner

KSmanis commented Aug 30, 2024

I'm not opposed to such a feature in principle, but my final say would depend on the maintenance burden. If the configuration is straightforward, CI is robust, and there are smoke tests, I would consider it for inclusion. If not, I guess you can maintain it in your fork or a downstream image.

Thanks!

henning-schild added a commit to henning-schild/docker-gentoo-distcc that referenced this issue Sep 4, 2024
henning-schild added a commit to henning-schild/docker-gentoo-distcc that referenced this issue Sep 4, 2024
henning-schild added a commit to henning-schild/docker-gentoo-distcc that referenced this issue Sep 4, 2024
henning-schild added a commit to henning-schild/docker-gentoo-distcc that referenced this issue Sep 4, 2024
henning-schild added a commit to henning-schild/docker-gentoo-distcc that referenced this issue Sep 19, 2024
@Goosegit11
Copy link

Goosegit11 commented Sep 23, 2024

I have a question - #66 (comment) ? How does ccache work with distcc? Do all nodes distribute their cache? and etc. would like to get an explanation of how it works

@henning-schild
Copy link
Contributor Author

Clients do not need their own cache. Should multiple clients send the same job to the same server, that server will serve from the cache instead of compiling again. Same would be true if you compiled the same thing multiple times from one client.

Clients can have their own caches as well. If a local cache would have a hit, the whole distcc machinery would not even be started. The local cache is probably best if you have one client and you want to speed up rebuilds or minor updates and have the space on your disk. A cache on the server is good if you have multiple machines which happen to be same CFLAGS and arch because you share work. Many combination are possible, including using one cache and sharing it via nfs.

You will have to see what works best for you, and will need time to learn that. Good sources of information will be the cache stats where you want a good ratio of hits, and qlop times where you want to see things getting faster.

@Goosegit11
Copy link

Goosegit11 commented Sep 24, 2024

Thanks! :-)

henning-schild added a commit to henning-schild/docker-gentoo-distcc that referenced this issue Sep 25, 2024
Instead of creating a new group put the user in the existing group. The
tcp version of the container will be running as distcc:distcc, using the
same group will allow sharing directories rw.

This patch prepares for the introduction of a persistent cache.

Related-to: KSmanis#64
Signed-off-by: Henning Schild <[email protected]>
henning-schild added a commit to henning-schild/docker-gentoo-distcc that referenced this issue Sep 25, 2024
henning-schild added a commit to henning-schild/docker-gentoo-distcc that referenced this issue Sep 25, 2024
Instead of creating a new group put the user in the existing group. The
tcp version of the container will be running as distcc:distcc, using the
same group will allow sharing directories rw.

This patch prepares for the introduction of a persistent cache.

Related-to: KSmanis#64
Signed-off-by: Henning Schild <[email protected]>
henning-schild added a commit to henning-schild/docker-gentoo-distcc that referenced this issue Sep 25, 2024
henning-schild added a commit to henning-schild/docker-gentoo-distcc that referenced this issue Sep 25, 2024
henning-schild added a commit to henning-schild/docker-gentoo-distcc that referenced this issue Sep 25, 2024
Instead of creating a new group put the user in the existing group. The
tcp version of the container will be running as distcc:distcc, using the
same group will allow sharing directories rw.

This patch prepares for the introduction of a persistent cache.

Related-to: KSmanis#64
Signed-off-by: Henning Schild <[email protected]>
henning-schild added a commit to henning-schild/docker-gentoo-distcc that referenced this issue Sep 25, 2024
henning-schild added a commit to henning-schild/docker-gentoo-distcc that referenced this issue Sep 25, 2024
henning-schild added a commit to henning-schild/docker-gentoo-distcc that referenced this issue Oct 1, 2024
Instead of creating a new group put the user in the existing group. The
tcp version of the container will be running as distcc:distcc, using the
same group will allow sharing directories rw.

This patch prepares for the introduction of a persistent cache.

Related-to: KSmanis#64
Signed-off-by: Henning Schild <[email protected]>
henning-schild added a commit to henning-schild/docker-gentoo-distcc that referenced this issue Oct 1, 2024
henning-schild added a commit to henning-schild/docker-gentoo-distcc that referenced this issue Oct 1, 2024
Instead of creating a new group put the user in the existing group. The
tcp version of the container will be running as distcc:distcc, using the
same group will allow sharing directories rw.

This patch prepares for the introduction of a persistent cache.

Related-to: KSmanis#64
Signed-off-by: Henning Schild <[email protected]>
henning-schild added a commit to henning-schild/docker-gentoo-distcc that referenced this issue Oct 1, 2024
henning-schild added a commit to henning-schild/docker-gentoo-distcc that referenced this issue Oct 1, 2024
henning-schild added a commit to henning-schild/docker-gentoo-distcc that referenced this issue Oct 1, 2024
KSmanis pushed a commit that referenced this issue Oct 12, 2024
Instead of creating a new group put the user in the existing group. The
tcp version of the container will be running as distcc:distcc, using the
same group will allow sharing directories rw.

This patch prepares for the introduction of a persistent cache.

Related-to: #64
Signed-off-by: Henning Schild <[email protected]>
KSmanis pushed a commit that referenced this issue Oct 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants