Use ansibleEE library instead of AnsibleEE v1 CR #972

bshephar · 2024-07-25T01:09:26Z

This change swaps out the use of the AnsibleEE custom resource for a generic Go library implementation. This enables us to stop using the unnecessary OpenStackAnsibleEE abstraction that we currently have around Kubernetes Jobs for AnsibleEE executions.

Jira: https://issues.redhat.com/browse/OSPRH-8926

openshift-ci · 2024-07-25T01:09:29Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

pkg/dataplane/util/ansible_execution.go

softwarefactory-project-zuul · 2024-07-25T01:16:58Z

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/8d8fc165435e40cd8a678ed706092259

❌ openstack-k8s-operators-content-provider FAILURE in 6m 37s
⚠️ podified-multinode-edpm-deployment-crc SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ cifmw-crc-podified-edpm-baremetal SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ adoption-standalone-to-crc-ceph-provider SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ openstack-operator-tempest-multinode SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider

softwarefactory-project-zuul · 2024-07-25T01:38:25Z

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/ff4f8dea98fc4e5db900f382d883aa83

❌ openstack-k8s-operators-content-provider FAILURE in 8m 16s
⚠️ podified-multinode-edpm-deployment-crc SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ cifmw-crc-podified-edpm-baremetal SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ adoption-standalone-to-crc-ceph-provider SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ openstack-operator-tempest-multinode SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider

softwarefactory-project-zuul · 2024-07-25T05:28:06Z

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/1ae4243f9a8e4f3b8f4679b20439a9f4

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 38m 49s
❌ podified-multinode-edpm-deployment-crc FAILURE in 1h 38m 49s
❌ cifmw-crc-podified-edpm-baremetal FAILURE in 1h 31m 20s
❌ adoption-standalone-to-crc-ceph-provider FAILURE in 2h 23m 53s
❌ openstack-operator-tempest-multinode FAILURE in 1h 44m 08s

bshephar · 2024-07-25T06:49:39Z

Looks like an issue with the GetAnsibleExecution():

2024-07-25T03:33:09.611Z        INFO    Controllers.OpenStackDataPlaneDeployment        ServiceRepoSetupDeploymentReady OpenStackAnsibleEE not yet found        {"controller": "openstackdataplanedeployment", "controllerGroup": "dataplane.openstack.org", "controllerKind": "OpenStackDataPlaneDeployment", "OpenStackDataPlaneDeployment": {"name":"edpm-deployment","namespace":"openstack"}, "namespace": "openstack", "name": "edpm-deployment", "reconcileID": "3c90b7e8-3a39-4224-9875-897d4ed3db08"}

softwarefactory-project-zuul · 2024-07-25T23:22:23Z

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.
Warning:
Error merging github.com/openstack-k8s-operators/openstack-operator for 972,15fa6e850a9cb6ca8efc0f23b92fd9a7e5b3c1e4

softwarefactory-project-zuul · 2024-07-29T00:26:21Z

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.
Warning:
Error merging github.com/openstack-k8s-operators/openstack-operator for 972,c6aad6f604b9b4d6f2cfef5edccfd03c3dfdad55

softwarefactory-project-zuul · 2024-07-29T00:29:30Z

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.
Warning:
Error merging github.com/openstack-k8s-operators/openstack-operator for 972,13a7570ac407ef70783c1cefee2bc89c2e31fe0e

softwarefactory-project-zuul · 2024-07-29T00:30:35Z

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.
Warning:
Error merging github.com/openstack-k8s-operators/openstack-operator for 972,5e7a6cf120950bc5c80961281433cfe154b9bb4c

softwarefactory-project-zuul · 2024-07-29T00:33:54Z

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.
Warning:
Error merging github.com/openstack-k8s-operators/openstack-operator for 972,3877b1fd084db964183b04bb3f7cc417888529de

go.mod

pkg/dataplane/util/ansible_execution.go

rabi · 2024-09-02T03:36:38Z

pkg/dataplane/deployment.go

-		if ansibleEE.Status.JobStatus == ansibleeev1.JobStatusRunning || ansibleEE.Status.JobStatus == ansibleeev1.JobStatusPending {
-			log.Info(fmt.Sprintf("AnsibleEE job is not yet completed: Execution: %s, Status: %s", ansibleEE.Name, ansibleEE.Status.JobStatus))
+		if ansibleEE.Status.Succeeded == 0 || ansibleEE.Status.Active != 0 && ansibleEE.Status.Failed != 0 {
+			log.Info(fmt.Sprintf("AnsibleEE job is not yet completed: Execution: %s, Active pods: %d", ansibleEE.Name, ansibleEE.Status.Active))


I think there would be scenario when all the three are zero (status.active == 0, status.succeeded == 0 and status.failed == 0) which is when the job is pending and we need to update the status with .readyWaitingMessage. I can't see how that is handled here.

pkg/dataplane/deployment.go

rabi

+2, could be refactored further to change some of the functions from GetAnsibleExecution to GetEEJob etc. Also, remove some of the comments like https://github.com/openstack-k8s-operators/openstack-operator/blob/main/pkg/dataplane/util/ansible_execution.go#L41.

bshephar · 2024-09-06T09:01:23Z

/test openstack-operator-build-deploy-kuttl

bshephar · 2024-09-07T03:53:39Z

/test openstack-operator-build-deploy-kuttl

bshephar · 2024-09-07T04:22:13Z

/test openstack-operator-build-deploy-kuttl

bshephar · 2024-09-07T22:02:42Z

/test openstack-operator-build-deploy-kuttl

bshephar · 2024-09-08T21:58:21Z

/retest-required

This change swaps out the use of the AnsibleEE custom resource for a generic Go library implementation. This enables us to stop using the unnecessary OpenStackAnsibleEE abstraction that we currently have around Kubernetes Jobs for AnsibleEE executions. Jira: https://issues.redhat.com/browse/OSPRH-8926 Signed-off-by: Brendan Shephard <[email protected]>

This change updates the functional tests to work with the changes from Ansibleeev1 to the new lib-common AnsibleEE library. Signed-off-by: Brendan Shephard <[email protected]>

Signed-off-by: Brendan Shephard <[email protected]>

softwarefactory-project-zuul · 2024-09-10T00:31:36Z

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/c5a51331398f44c780df58ad6829e5dc

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 55m 05s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 18m 14s
❌ cifmw-crc-podified-edpm-baremetal RETRY_LIMIT in 21m 37s
✔️ openstack-operator-tempest-multinode SUCCESS in 1h 40m 56s

bshephar · 2024-09-10T00:32:39Z

recheck

softwarefactory-project-zuul · 2024-09-10T00:38:45Z

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/3fcaaf93f1984609b949078d6856fbda

❌ openstack-k8s-operators-content-provider FAILURE in 5m 03s
⚠️ podified-multinode-edpm-deployment-crc SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ cifmw-crc-podified-edpm-baremetal SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ openstack-operator-tempest-multinode SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider

bshephar · 2024-09-10T00:42:09Z

recheck

bshephar · 2024-09-10T07:10:06Z

+2, could be refactored further to change some of the functions from GetAnsibleExecution to GetEEJob etc. Also, remove some of the comments like https://github.com/openstack-k8s-operators/openstack-operator/blob/main/pkg/dataplane/util/ansible_execution.go#L41.

Yeah, the file could use some work. I have another branch with some refactoring here:
dae8a70

Once this one merges, I'll be able to tidy it up and send that branch as a PR to clean up the rest of it. This PR was already too big and too many changes, so I left any additional refactoring for another PR.

openshift-ci · 2024-09-10T10:14:20Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bshephar, fao89

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [bshephar,fao89]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

fao89 · 2024-09-10T10:15:45Z

/hold

fao89 · 2024-09-10T10:21:58Z

hold wasn't feast enough 😓
I wanted to double-check first if we could merge it, but the bot was faster

bshephar · 2024-09-10T21:20:39Z

It was ready. I have some further refactoring in another branch. So I'll send that and we can make any additional adjustments as part of that PR.

openshift-ci bot added do-not-merge/work-in-progress approved labels Jul 25, 2024

bshephar commented Jul 25, 2024

View reviewed changes

pkg/dataplane/util/ansible_execution.go Outdated Show resolved Hide resolved

bshephar force-pushed the ansibleee-library branch from 696c586 to ea26612 Compare July 25, 2024 01:29

bshephar force-pushed the ansibleee-library branch 2 times, most recently from 179aded to 6bf7c55 Compare July 25, 2024 02:48

openshift-merge-robot added the needs-rebase label Jul 25, 2024

bshephar force-pushed the ansibleee-library branch from 6bf7c55 to 15fa6e8 Compare July 25, 2024 23:22

bshephar force-pushed the ansibleee-library branch from 15fa6e8 to c6aad6f Compare July 29, 2024 00:26

bshephar force-pushed the ansibleee-library branch from c6aad6f to 13a7570 Compare July 29, 2024 00:29

bshephar force-pushed the ansibleee-library branch from 13a7570 to 5e7a6cf Compare July 29, 2024 00:30

bshephar force-pushed the ansibleee-library branch from 5e7a6cf to 3877b1f Compare July 29, 2024 00:33

bshephar force-pushed the ansibleee-library branch from 3877b1f to 0e6b954 Compare July 29, 2024 05:55

openshift-merge-robot removed the needs-rebase label Jul 29, 2024

bshephar commented Jul 29, 2024

View reviewed changes

go.mod Outdated Show resolved Hide resolved

bshephar commented Jul 29, 2024

View reviewed changes

pkg/dataplane/util/ansible_execution.go Outdated Show resolved Hide resolved

bshephar force-pushed the ansibleee-library branch 4 times, most recently from e0f2fe7 to eb2fb61 Compare August 1, 2024 04:46

bshephar force-pushed the ansibleee-library branch from d1a3572 to 8e48cf0 Compare September 2, 2024 00:48

rabi reviewed Sep 2, 2024

View reviewed changes

bshephar force-pushed the ansibleee-library branch from 8e48cf0 to 62d186a Compare September 2, 2024 10:57

rabi reviewed Sep 3, 2024

View reviewed changes

bshephar force-pushed the ansibleee-library branch from 62d186a to 6bc57fa Compare September 3, 2024 07:23

bshephar added 4 commits September 10, 2024 08:35

Update functional tests for ansible lib

16b76b7

This change updates the functional tests to work with the changes from Ansibleeev1 to the new lib-common AnsibleEE library. Signed-off-by: Brendan Shephard <[email protected]>

Remove AnsibleEE Operator

b54fc03

Signed-off-by: Brendan Shephard <[email protected]>

Allow for Jobs instead of AnsibleEE CR in kuttl

0ac1288

Signed-off-by: Brendan Shephard <[email protected]>

bshephar force-pushed the ansibleee-library branch from 6bc57fa to 0ac1288 Compare September 9, 2024 22:35

fao89 approved these changes Sep 10, 2024

View reviewed changes

openshift-ci bot assigned fao89 Sep 10, 2024

openshift-ci bot added the lgtm label Sep 10, 2024

openshift-ci bot added the do-not-merge/hold label Sep 10, 2024

openshift-merge-bot bot merged commit fdde812 into openstack-k8s-operators:main Sep 10, 2024
7 checks passed

bshephar mentioned this pull request Sep 10, 2024

Ansibleee refactor #1059

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use ansibleEE library instead of AnsibleEE v1 CR #972

Use ansibleEE library instead of AnsibleEE v1 CR #972

bshephar commented Jul 25, 2024

openshift-ci bot commented Jul 25, 2024

softwarefactory-project-zuul bot commented Jul 25, 2024

softwarefactory-project-zuul bot commented Jul 25, 2024

softwarefactory-project-zuul bot commented Jul 25, 2024

bshephar commented Jul 25, 2024

softwarefactory-project-zuul bot commented Jul 25, 2024

softwarefactory-project-zuul bot commented Jul 29, 2024

softwarefactory-project-zuul bot commented Jul 29, 2024

softwarefactory-project-zuul bot commented Jul 29, 2024

softwarefactory-project-zuul bot commented Jul 29, 2024

rabi Sep 2, 2024

rabi left a comment

bshephar commented Sep 6, 2024

bshephar commented Sep 7, 2024

bshephar commented Sep 7, 2024

bshephar commented Sep 7, 2024

bshephar commented Sep 8, 2024

softwarefactory-project-zuul bot commented Sep 10, 2024

bshephar commented Sep 10, 2024

softwarefactory-project-zuul bot commented Sep 10, 2024

bshephar commented Sep 10, 2024

bshephar commented Sep 10, 2024

openshift-ci bot commented Sep 10, 2024

fao89 commented Sep 10, 2024

fao89 commented Sep 10, 2024

bshephar commented Sep 10, 2024

Use ansibleEE library instead of AnsibleEE v1 CR #972

Use ansibleEE library instead of AnsibleEE v1 CR #972

Conversation

bshephar commented Jul 25, 2024

openshift-ci bot commented Jul 25, 2024

softwarefactory-project-zuul bot commented Jul 25, 2024

softwarefactory-project-zuul bot commented Jul 25, 2024

softwarefactory-project-zuul bot commented Jul 25, 2024

bshephar commented Jul 25, 2024

softwarefactory-project-zuul bot commented Jul 25, 2024

softwarefactory-project-zuul bot commented Jul 29, 2024

softwarefactory-project-zuul bot commented Jul 29, 2024

softwarefactory-project-zuul bot commented Jul 29, 2024

softwarefactory-project-zuul bot commented Jul 29, 2024

rabi Sep 2, 2024

Choose a reason for hiding this comment

rabi left a comment

Choose a reason for hiding this comment

bshephar commented Sep 6, 2024

bshephar commented Sep 7, 2024

bshephar commented Sep 7, 2024

bshephar commented Sep 7, 2024

bshephar commented Sep 8, 2024

softwarefactory-project-zuul bot commented Sep 10, 2024

bshephar commented Sep 10, 2024

softwarefactory-project-zuul bot commented Sep 10, 2024

bshephar commented Sep 10, 2024

bshephar commented Sep 10, 2024

openshift-ci bot commented Sep 10, 2024

fao89 commented Sep 10, 2024

fao89 commented Sep 10, 2024

bshephar commented Sep 10, 2024