Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enabling ccache on github builds for Ubuntu #95

Merged
merged 51 commits into from
May 17, 2021
Merged

Conversation

jerinphilip
Copy link
Contributor

@jerinphilip jerinphilip commented Apr 9, 2021

Base automatically changed from jp/brt-ghci to main April 16, 2021 10:58
@XapaJIaMnu
Copy link
Collaborator

WHat is the status of this?

@jerinphilip
Copy link
Contributor Author

jerinphilip commented Apr 22, 2021

@XapaJIaMnu Cache (CCACHE based) is working, but I don't see any speed gains. I also had plans to cache the build folder (it's ready in theory, duplicate code). The host-os keeps changing between avx2 and avx512 so unsure how reusable this is.

Is this related?

@XapaJIaMnu
Copy link
Collaborator

No, this is completely unrelated. But since we're compiling with march native CCache would be unusable in between machines with different signatures.

@kpu
Copy link
Member

kpu commented Apr 22, 2021

@XapaJIaMnu
Copy link
Collaborator

Shall we close this?

@jerinphilip
Copy link
Contributor Author

@XapaJIaMnu No, we'll keep this. I'm reading more into this ccache business over multiple C++ projects which use GitHub CI, still hopeful something can be done.

@kpu
Copy link
Member

kpu commented May 10, 2021

GitHub actions is this mysterious poorly documented thing to me. @graemenail said he does continuous distribution of his CV on his CV, so I'm passing the review over to him.

@XapaJIaMnu
Copy link
Collaborator

What speedup are we seeing from CC cache?

@jerinphilip
Copy link
Contributor Author

@XapaJIaMnu 5m, 3m for what was 30m, 17m previously if cache hit. They're in the screenshots. I think best for us is to drop march=native and fix the instruction to ssse3 or avx2 for cache advantages.

@XapaJIaMnu
Copy link
Collaborator

I am absolutely in favour of keeping native because we have had breakages in the past the involve only newer ISA. We should indeed have an option that limits the architecture too. This is slightly more complicated because the way the architecture is set on Linux/Mac is different from the way its set on Windows.

@XapaJIaMnu
Copy link
Collaborator

I'll wait for @graemenail 's review

CMakeLists.txt Outdated Show resolved Hide resolved
Copy link
Member

@graemenail graemenail left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Would be interested to see the cache usage if one were to re-run the job from UI.

.github/workflows/native-ubuntu.yml Outdated Show resolved Hide resolved
.github/workflows/native-ubuntu.yml Show resolved Hide resolved
scripts/ci/compiler-hash.sh Show resolved Hide resolved
CMakeLists.txt Outdated Show resolved Hide resolved
scripts/ci/compiler-hash.sh Show resolved Hide resolved
@jerinphilip
Copy link
Contributor Author

Would be interested to see the cache usage if one were to re-run the job from UI.

@graemenail 10m, 20m vs 2m, 5m for minimal-marian and full-marian respectively, performed on 3b8205d (#95).

Normal cache run

image

UI Rerun

image

@jerinphilip jerinphilip requested a review from XapaJIaMnu May 14, 2021 11:42
@XapaJIaMnu XapaJIaMnu merged commit 77424a3 into main May 17, 2021
@jerinphilip jerinphilip deleted the jp/brt-ghci-ccache branch May 18, 2021 19:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants