Migrate the Packages_Builder_AMI pipeline to GHA

[teddytpc1]

Objective
wazuh/wazuh-packages#2904

Description

Because of the Wazuh packages redesign tier 2 objective we need to migrate the Packages_Builder_AMI pipeline from Jenkins to GHA.

Tasks

• Create a directory for the AMI in this repository
• Replicate the Packages_Builder_AMI pipeline in GHA
• The GHA should allow the following parameters:
  • PACKAGE_VERSION
  • VIRTUAL_MACHINES_REFERENCE
  • FILEBEAT_VERSION
  • S3_BUCKET
  • PACKAGES_REPOSITORY
  • S3_REPOSITORY
• The GHA should allow to be executed on demand
• Validate the AMI generated by the GHA works

Related

• OVA and AMI code and pipelines migration #22

[Enaraque]

Update report

I've been learning how the previous Jenkins pipeline worked and researching how to adapt it to GitHub Actions.

For now, it would be necessary to use secrets for access to AWS (access_key and secret_access_key). This won't be necessary if it’s decided to use AWS's OIDC with GitHub in the end.

Next approach

Continue researching and implement part of the required code.

[Enaraque]

Update report

We have been working on the AMI build workflow.

For now, the first version has been implemented, but testing still needs to be done.

The playbook needed to install the Wazuh components has also been added.

[Enaraque]

Update report

Once the workflow has been uploaded, tests are being carried out for its correct implementation.

[Enaraque]

Update report

Various tests have been conducted with the workflow. The main issues are related to the execution of the playbook for installing Wazuh components on the generated instance.

The primary problems arise from the wazuh-user not being created on the instance. To address this, the playbook wazuh_user_configuration.yaml has been implemented, which creates the wazuh-user and grants it the necessary permissions.

Using a fresh instance instead of the AMI previously used with Jenkins has also caused issues with modifying files in the wazuh-user home directory, which were not created.

Currently, efforts are focused on resolving problems related to this user to move forward with the workflow execution.

[Enaraque]

Update report

The workflow is executed as expected. These tests have been done on a base AL2.

The image used to build the AMI has been added to the allocator so that it can be built on it from now on, this solves the problems with the wazuh-user.

Next approach

Test that the workflow runs correctly on the image uploaded to the allocator. In addition, one more step needs to be added to the workflow in order to obtain the corresponding artefact.

[Enaraque]

Update report

The option to save an artifact containing the necessary information for the allocator has been implemented. This step is executed whenever the base instance is created and the "destroy" option is disabled. This allows us to obtain a file with the necessary information to access the instance if desired.
Example workflow: https://github.com/wazuh/wazuh-virtual-machines/actions/runs/10668650810

The destroy step has also been modified so that it executes whenever (the destroy option must be marked):

• The instance is created, but the pipeline fails or is canceled
• The pipeline completes successfully In each case

It has been verified that the pipeline executes correctly with the image that was previously used in the Jenkins pipeline to generate the image.
Example (before adding the destroy steps above): https://github.com/wazuh/wazuh-virtual-machines/actions/runs/10662630686

Once the pipeline is finished, the dashboard can be accessed correctly.

Next approach

There are a few more deployments to test but no more code to implement.

[Enaraque]

Update report

After running some workflows testing different behaviors, and testing the AMI generated, every run as expected.

Tests

Workflow execution ends succesfully

With destroy option

The artifact is not generated and the instance is deleted:

Without destroy option

The artifact was successfully generated and the instance is not deleted:

Workflow don't finish (canceled)

Here we test if the base instance is deleted if the destroy options is set, and if the artifact is generated when is needed.

Base instance is created and the destroy option is not set

The artifact was successfully generated and the instance is not deleted:

Base instance created and the destroy option is set

The artifact is not generated and the instance is deleted:

Base instance is not created and the destroy option is set

The instance is not deleted because it has not been generated and the artifact is not generated:

Instance created with the AMI

Once the AMI has been created, if we create an instance with that image, we can see that the dashboard can be accessed correctly.

Instance and Dashboard access

[Enaraque]

The parameters to use in the workflow are the following:

• WAZUH_VIRTUAL_MACHINES_REFERENCE
• WAZUH_AUTOMATION_REFERENCE
• WAZUH_INSTALLATION_ASSISTANT_REFERENCE
• VERBOSITY
• SUFFIX_AMI
• DESTROY

Instead of (commented in the issue description):

• PACKAGE_VERSION
• VIRTUAL_MACHINES_REFERENCE
• FILEBEAT_VERSION
• S3_BUCKET
• PACKAGES_REPOSITORY
• S3_REPOSITORY

These parameters were changed as some of them were no longer needed. In addition, other necessary parameters were added such as VERBOSITY or DESTROY that appeared in the Jenkins pipeline or references to new repositories, necessary for the execution of the workflow.