Sccache doesn't seem to work in CI

[Kailai-Wang]

Context

I notice that sccache doesn't seem to take effect, an example: https://github.com/litentry/litentry-parachain/actions/runs/3405980852

Even if I re-run the job without changing anything, where I expect the run-cargo-runtime-tests to complete quickly, it still did a full compilation.

Strangely, the cargo registry and sccache cache were restored successfully:

Cache restored successfully
Cache restored from key: cargo-7cca616f672081f386280a7525ace5640c511a4aebda45281f073c7e8c5e3007

Cache restored successfully
Cache restored from key: sccache-7cca616f672081f386280a7525ace5640c511a4aebda45281f073c7e8c5e3007

It just didn't get applied.

In the sccache stat, it showed:

Note the cache hit were all C/C++.

Once it's solved, the next step is to try to use sccache inside docker

---

✔️ Please set appropriate labels and assignees if applicable.

[Kailai-Wang]

Seemingly it's due to the hashing key of sccache for the cargo-unittest: hash of Cargo.lock files are not enough, especially for the incremental building of the same feature branch.

We can have finer control over it by adjusting the cache key: including the branch/tag name looks helpful: https://github.com/litentry/litentry-parachain/actions/runs/3409984123/jobs/5672416950#step:9:16

For PRs we use the github.base_ref as part of the key trying to increase the accuracy of the cache

[Kailai-Wang]

sccache inside docker looks not bad:

[Kailai-Wang]

but please note it only works with the consistent runner (self-hosted).

Conclusion so far regarding caching:

• sccache inside docker for self-hosted runner
• cargo-chef + GHA gha/local cache for github runner