Implement multi runner process support for runner scale down.

[krlydm]

Current scale down implementation does not support the use case when multiple GH runner processes running on an instance, which can be a valid use case. This change is handling this on scale down lambda side, so whenever a prebuilt image is configured to start multiple runner processes on an instance, scale down can handle the idle/active situation on a single instance by checking busy state of the runner. The only restriction is that whenever multiple GH runner processes started their name must start with the instance id (as like currently GH runner name is the instance ID).

This change is a possible fix to solve the issue reported in #166.

Feel free to question or reject my change, I was thinking about opening a new issue, but it was easier to implement than ask. I'm open to discuss this further.

[npalm]

@krlydm please can you rebase this PR?

[npalm]

Could you share the setup to run multiple runners on a single instance as an example in a PR as well?

[krlydm]

Could you share the setup to run multiple runners on a single instance as an example in a PR as well?

I'm using custom install and setup scripts to configure multiple runners, do you want me to create a new packer template under images or I even thinking about introducing a new config, something like instance_number_of_runners, store it in SSM, and update the existing start-runner scripts to use this config properly. This new config can be also used to make scale up smarter, so if we have 2 job to execute from GH and instance is configured to run 2 runners, then scale-up only starts one instance. What do you think?

[npalm]

@krlydm Sorry for keep you waiting, but need some time to test an actual deployment of this PR. Hope to be able to run a test early next week. Sorry for the delay. Thanks for the contribution!

[npalm]

@krlydm sorry still not time, the PR is still on my radar!

[npalm]

@krlydm looks good, thanks!

[npalm]

close #166

[rashidnhm]

I'd like to use this new feature myself ... I had one question @krlydm ... when setting up new runners with multiple installations are you able to re-use the same token stored on SSM? Currently the scale-up puts the required url and token as a SecureString in SSM. And would you have a way to share your packer templates with multiple installation for me to take a look at?

[krlydm]

I'd like to use this new feature myself ... I had one question @krlydm ... when setting up new runners with multiple installations are you able to re-use the same token stored on SSM? Currently the scale-up puts the required url and token as a SecureString in SSM. And would you have a way to share your packer templates with multiple installation for me to take a look at?

You have to have your own configure and start runner scripts, which downloads the GitHub runner and extracts it to folders based on how much runner process you want. I'm using the PowerShell scripts as an example, but same thing can be applied for linux as well. Based on the Windows image example, the following script is doing this:

terraform-aws-github-runner/images/windows-core-2019/windows-provisioner.ps1

Line 43 in 3658d6a

Expand-Archive -Path actions-runner.zip -DestinationPath .

The change you have to do is to place a for loop here an exact runners in a numbered folder, like C:\actions-runner\1, C:\actions-runner\2, etc.

for ($i = 1; $i -le 2; $i++)
{ 
    New-Item -ItemType Directory -Path C:\actions-runner\$i 
    Write-Host "Un-zip action runner"
    Expand-Archive -Path actions-runner.zip -DestinationPath ./$i
}

Once you have the configuration part, you need an own setup script, because a similar for loop is needed there, GitHub runner config and run calls are needed to be called on each numbered folder with the proper agent naming:

for ($i = 1; $i -le 2; $i++)
{
    Set-Location C:\actions-runner\$i 
    $configCmd = ".\config.cmd --unattended --name $InstanceId-$i --work `"_work`" $config"
    Write-Host "Configure GH Runner as user $run_as"
    Invoke-Expression $configCmd
     ...
}

terraform-aws-github-runner/modules/runners/templates/start-runner.ps1

Line 85 in 3658d6a

$configCmd = ".\config.cmd --unattended --name $InstanceId --work `"_work`" $config"

[rashidnhm]

Ah I see! And you were able to re-use the same $config for starting multiple instances of the runner. Thank you very much for this!