chore(cleanup AWS) garbage collect EC2 snapshots older than 1 year on us-east-2

[dduportal]

Implements jenkins-infra/helpdesk#2846 (comment), in the context of jenkins-infra/helpdesk#3502.

This PR add a GC for AWS snapshots, restricted to us-east-2 .

[dduportal]

Initial build was too long (more than 20 min and counting). Let's start slowly with a 12month threshold for the first cleanup operation.

[dduportal]

• With the "1 year" threshold, the dry run GC for snapshots took ~12 min. Let's start with this value.
• Stopped the build (as no need to consume VMs for this script to run).

[lemeurherve]

LGTM, wondering if regrouping the two scripts together would worth it 🤔
(It seems to me they are almost similar except for the query, the cleanup command parameter and the subject of the log messages)

[lemeurherve]

LGTM
Interested to know nonetheless WDYT about #587 (review)

[dduportal]

LGTM, wondering if regrouping the two scripts together would worth it 🤔 (It seems to me they are almost similar except for the query, the cleanup command parameter and the subject of the log messages)

The goal is to keep a list of simpler scripts to be run independantly of each other.
There are also slight difference between commands: we could factorize but it's not the effort on my opinion

[dduportal]

This PR, once applied, failed the GC with the following error:

An error occurred (InvalidSnapshot.InUse) when calling the DeleteSnapshot operation: The snapshot snap-0639dfcc718eaf8db is currently in use by ami-0995d04457d9af25e

• Build continues so not a blocker, but no GC is applied to snapshots :'(
• Incoming PRs:
  • Parallellize (and avoid fail fast) when deleting images to fix the above error
  • Cleaning up staging images (more deregistered AMIs)
  • Cleaning up prod images (more deregistered AMIs)