chore: ensure that spot instances are used for packer builds

[dduportal]

As per jenkins-infra/helpdesk#3502 and jenkins-infra/helpdesk#3485, cloud cost control is important for us.

The goal of this PR is to ensure that packer VM used to build VM templates are always using Spot prices on both AWS EC2 and Azure.

[smerle33]

there is an error with the spot

[dduportal]

Weird, it points to the spot = true in the azure-arm source, which I got from https://developer.hashicorp.com/packer/plugins/builders/azure/arm#spot .

I might be mis-reading the doc, lemme try locally

[dduportal]

Oh I see, there documentation is incomplete: https://github.com/hashicorp/packer-plugin-azure/blob/79cd1eab80ead3910cf7507ef8355766c816818d/builder/azure/arm/config_test.go#L2644-L2646

[dduportal]

Opened a PR on their doc: hashicorp/packer-plugin-azure#293

[dduportal]

Still in error: it's for the AWS build this time

==> amazon-ebs.ubuntu: Error launching source instance: InvalidParameterValue: Invalid value '' for InstanceType.

[dduportal]

• AWS spot problem reproduced locally
• We are hitting using spot_instance_types returns InvalidParameterValue: Invalid value '' for InstanceType. hashicorp/packer-plugin-amazon#198
• Adding the spot_price atrribute explictly set to auto allows me to spin up an EC2 arm64 VM, so let's try this on the CI now

[dduportal]

I'm force merging under the following assumptions:

• It can't spin up an ARM64 VM (in EC2) for the ARM64 Docker build, because I might have been too "hasty" in cleaning up AMIs in EC2 (error com.amazonaws.AmazonClientException: Unable to find image for request {ExecutableUsers: [],Filters: [],ImageIds: [ami-0a0af3a09497058f7],Owners: [200564066411],} in infra.ci logs)
  => 1.1.1 release is needed

• The EC2 and Azure VM builds worked well which validate this PR