Investigate the use of github caches to speed up ci runs

[brendanheywood]

Before any of this is started get some really good way to get average metrics for some plugins, ideally with a history of all the run times. Even if this is just a spreadsheet somewhere and we dump stuff into it.

This should save a good chunk of ci setup time for the moodle checkout, npm, composer and more

https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows

• npm

https://github.com/actions/setup-node#caching-global-packages-data

• Composer

https://github.com/shivammathur/setup-php/blob/758e69ee603cd38b121aeac6e0b0e02e8d2a0ab9/examples/cakephp.yml#L24-L35

• shallow clone of moodle stables (maybe download and cache the zip from github instead of git clone?)
  Clone takes about 19 seconds, download zip around 11, unzip around 3. So this is probably a marginal win

[keevan]

I think this would be good, however it might only work within the same moodle branch versions (as dependencies are likely to change between versions).

That said, it still might be good since there are a couple of different combinations that run on LTS branches, especially with the possibility of adding another DB (mysql) into he mix as per #65

It would be good to also see / know some rough timings mentioned for the npm/composer installs, because it feels like they shouldn't take that long to install in the grand scheme of things (e.g. a moodle db install takes longer)

[brendanheywood]

Ok maybe a crazy idea. We spend quite a bit of time installing moodle. What if we:

1. checked if there was a db backup in the cache
2. if not we install a clean moodle
3. then we back up the db and put it into the cache
4. then we checkout our plugin and run the upgrade on that

The clean moodle backup would have a key hash based on the version so it should be valid for days to weeks.

[danmarsden]

Would that pick up install failures? I've seen plugins fail to install due to various things but work fine when upgrading.

[brendanheywood]

What I proposed above would not have the plugin installed in the db backup, this is so the db file can be safely cached using just the core version or git commit hash as its key and still get lots of cache hits across runs. If the cache had the plugin code included then it would almost never get a cache hit.

Also assuming the plugin was installed and it was in the cache, then you could potentially end up with the reverse problem which is the upgrade script is correct but the install.xml file is incorrect. If it worked (cold cache so installed) and then stopped working (upgrade is wrong) or vice versa then most peoples knee jerk reaction would be that the ci is flaky.

The only way to properly solve that is a separate check devoted to testing the upgrade paths for consistency, ie check out some arbitrarily old version, install it, and then run all of the upgrade steps and assert the schema check is green and in the same state as a fresh install from the latest version. We logged this yonks ago before hq took over the plugin here:

open-lms-open-source/moodle-plugin-ci#96

Now that hq has this moving again we should resubmit that in the right place:

https://github.com/open-lms-open-source/moodle-plugin-ci/pull/102/files

That above is only the check schema test, not the whole solution

Some relationship with this one: moodlehq/moodle-plugin-ci#156

[brendanheywood]

Just made that here:

moodlehq/moodle-plugin-ci#180

[danmarsden]

Sure - what I meant was...I've seen plugins fail to install on a fresh moodle (install plugin at same time as moodle install) bur work fine if moodle is installed then the plugin added later. We should still test that imo.