Runner package is downloaded every single time

[DavidGOrtega]

Would be faster if we can reuse the already downloaded one, this is specially good for not cloud runners.
We should download the wrapped runner once (GH or GL), install it in the system and make cml-runner to use it always.
If we are changing the SCM we should download the other wrapped runner.

[0x2b3bfa0]

See the discussion at #580 for context.

Definitely, downloading the self-hosted runner package every single time is suboptimal, but requiring users to download two or more different runner packages along with CML when they just need one or none is equally inconvenient. It would be nice to cache them locally, though.

[0x2b3bfa0]

Just to justify the good first issue label, runner packages are being downloaded from here:

cml/src/drivers/github.js

Line 197 in 6c451d9

const url = `https://github.com/actions/runner/releases/download/v${ver}/actions-runner-${arch}-${ver}.tar.gz`;

cml/src/drivers/gitlab.js

Lines 153 to 154 in 6c451d9

	const url =
	'https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64';

[DavidGOrtega]

Definitely, downloading the self-hosted runner package every single time is suboptimal, but requiring users to download two or more different runner packages along with CML when they just need one or none is equally inconvenient.

I do not follow you here. So Im going to edit the main description.

[0x2b3bfa0]

I'm going to edit the main description.

Thanks! Now it's much more clear.

[SuTran]

Hi @0x2b3bfa0,

I plan to take this, how do you think about the below approach:

I will created a cached folder ${homedir()}/.cml/cached to store the action-runner.tar.gz and add an utils method called checkFileExists to utils.js. I wonder if we need to invalidate the cache sometime?

[dacbd]

is this still a concern/issue? Im not following what the optimization is supposed to be?

[0x2b3bfa0]

This is only an “issue” when using cml runner locally; i.e. no --cloud

[0x2b3bfa0]

Definitely not p1-important

[dacbd]

I see because each call downloads the gha runner since they aren't aware of previous invocations

[jancrichter]

Are there any news on this?

Downloading the gitlab runner binary (50MB) every single time a runner is created considerably slows our pipelines because it has to go through our proxy, where it is scanned, every single time. This means an extra few minutes between triggering runner creation and start of the actual workload.

[dacbd]

@jancrichter no news on this; unfortunately, it's not yet on the team's priority list. However, If you are willing to do a PR and implement this feature, we would gladly welcome the contribution and will work to get it merged and released asap for you.