New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Meta content provider job to test opendev and github changes #1892

Merged

openshift-merge-bot merged 1 commit into main from meta_content_provider

Jun 28, 2024

Contributor

raukadah commented Jun 18, 2024 •

edited

Loading

Meta content provider does following things:

It combines 3 content provider (operator, tcib and edpm image build) into one content provider to test all changes together.
If the changes coming from opendev, tcib and os-net-config, then it will build the DLRN packages and create gating repo.
If there is a gating and non os-net-config changes, then it will do the tcib build.
If there is a edpm-image-build change, then it will perform the image build.
All the built content will be pushed on single registry with different namespace.
The content provider will expose the same zuul return vars to dependent job to pick proper built content.
It also returns content_provider_os_registry_url var to dependent job and improve edpm_prepare role to set proper vars.
Generate gating repo only when there are changes to built by DLRN.

Note: It make changes to edpm_prepare to check edpm_image_build_output key length. If no edpm image is built by meta content provider then edpm_image_build_output will return an empty dict.

Testing scenario:

meta-content-provider test results on edpm-image-builder repo
Opendev + operator changes tested together
Testing master opendev patches

As a pull request owner and reviewers, I checked that:

Appropriate testing is done and actually running

openshift-ci bot added the do-not-merge/work-in-progress label

Contributor

openshift-ci bot commented Jun 18, 2024

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

raukadah added a commit to raukadah/tcib that referenced this pull request


          [WIP] Testing meta content provider

ab8ae9a

Depends-On: https://review.opendev.org/c/openstack/nova/+/921686
Depends-On: openstack-k8s-operators/nova-operator#777
Depends-On: openstack-k8s-operators/ci-framework#1892

Signed-off-by: Chandan Kumar <[email protected]>

raukadah mentioned this pull request

Use Meta content provider in tcib openstack-k8s-operators/tcib#179

Merged

raukadah force-pushed the meta_content_provider branch from dc5c241 to 38ca095 Compare

June 18, 2024 08:22

raukadah added a commit to raukadah/tcib that referenced this pull request


          [WIP] Testing meta content provider

79db90c

Depends-On: https://review.opendev.org/c/openstack/nova/+/921686
Depends-On: openstack-k8s-operators/nova-operator#777
Depends-On: openstack-k8s-operators/ci-framework#1892

Signed-off-by: Chandan Kumar <[email protected]>

raukadah added a commit to raukadah/tcib that referenced this pull request


          [WIP] Testing meta content provider

2a33cf7

Depends-On: https://review.opendev.org/c/openstack/nova/+/921686
Depends-On: openstack-k8s-operators/nova-operator#777
Depends-On: openstack-k8s-operators/ci-framework#1892

Signed-off-by: Chandan Kumar <[email protected]>

raukadah force-pushed the meta_content_provider branch from 38ca095 to 43ffd5a Compare

June 18, 2024 09:09

raukadah added a commit to raukadah/tcib that referenced this pull request


          [WIP] Testing meta content provider

bce22b1

Depends-On: https://review.opendev.org/c/openstack/nova/+/921686
Depends-On: openstack-k8s-operators/nova-operator#777
Depends-On: openstack-k8s-operators/ci-framework#1892

Signed-off-by: Chandan Kumar <[email protected]>

raukadah force-pushed the meta_content_provider branch from 43ffd5a to a297950 Compare

June 18, 2024 10:06

raukadah added a commit to raukadah/tcib that referenced this pull request


          [WIP] Testing meta content provider

56347aa

Depends-On: https://review.opendev.org/c/openstack/nova/+/921686
Depends-On: openstack-k8s-operators/nova-operator#777
Depends-On: openstack-k8s-operators/ci-framework#1892

Signed-off-by: Chandan Kumar <[email protected]>

raukadah added a commit to raukadah/tcib that referenced this pull request


          [WIP] Testing meta content provider

121a5aa

Depends-On: https://review.opendev.org/c/openstack/nova/+/921687
Depends-On: openstack-k8s-operators/nova-operator#777
Depends-On: openstack-k8s-operators/ci-framework#1892

Signed-off-by: Chandan Kumar <[email protected]>

raukadah added a commit to raukadah/tcib that referenced this pull request


          [WIP] Testing meta content provider

4f5a21d

Depends-On: https://review.opendev.org/c/openstack/nova/+/921687
Depends-On: openstack-k8s-operators/nova-operator#777
Depends-On: openstack-k8s-operators/ci-framework#1892

Signed-off-by: Chandan Kumar <[email protected]>

softwarefactory-project-zuul bot commented Jun 18, 2024

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/bec90fd19fef4571a1303bfa3d360c72

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 57m 54s
❌ podified-multinode-edpm-deployment-crc POST_FAILURE in 1h 22m 23s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 11m 39s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 55s
❌ cifmw-pod-pre-commit FAILURE in 7m 42s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 42s

raukadah force-pushed the meta_content_provider branch from a297950 to f2db6f5 Compare

June 18, 2024 13:20

raukadah added a commit to raukadah/tcib that referenced this pull request


          [WIP] Testing meta content provider

2699f91

Depends-On: https://review.opendev.org/c/openstack/nova/+/921687
Depends-On: openstack-k8s-operators/nova-operator#777
Depends-On: openstack-k8s-operators/ci-framework#1892

Signed-off-by: Chandan Kumar <[email protected]>

raukadah added a commit to raukadah/tcib that referenced this pull request


          [WIP] Testing meta content provider

3ecc2ec

Depends-On: https://review.opendev.org/c/openstack/nova/+/921687
Depends-On: openstack-k8s-operators/nova-operator#777
Depends-On: openstack-k8s-operators/ci-framework#1892

Signed-off-by: Chandan Kumar <[email protected]>

softwarefactory-project-zuul bot commented Jun 18, 2024

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/69dd2963d4674105962eebb8825d296f

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 17m 00s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 17m 24s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 15m 58s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 44s
❌ cifmw-pod-pre-commit FAILURE in 7m 24s
✔️ cifmw-pod-zuul-files SUCCESS in 5m 00s

raukadah force-pushed the meta_content_provider branch from f2db6f5 to 00ba4cb Compare

June 19, 2024 06:16

raukadah added a commit to raukadah/tcib that referenced this pull request


          [WIP] Testing meta content provider

8c5987c

Depends-On: https://review.opendev.org/c/openstack/nova/+/921687
Depends-On: openstack-k8s-operators/nova-operator#777
Depends-On: openstack-k8s-operators/ci-framework#1892

Signed-off-by: Chandan Kumar <[email protected]>

softwarefactory-project-zuul bot commented Jun 19, 2024

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/4dc2042b7d45445b9b71423d074a7154

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 47m 15s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 21m 21s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 31m 51s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 7m 48s
✔️ cifmw-pod-pre-commit SUCCESS in 7m 27s
✔️ cifmw-pod-zuul-files SUCCESS in 5m 30s
❌ ci-framework-openstack-meta-content-provider FAILURE in 50m 43s

raukadah force-pushed the meta_content_provider branch from 00ba4cb to 6ed5a2b Compare

June 19, 2024 09:05

raukadah added a commit to raukadah/tcib that referenced this pull request


          [WIP] Testing meta content provider

d470a2b

Depends-On: https://review.opendev.org/c/openstack/nova/+/921687
Depends-On: openstack-k8s-operators/nova-operator#777
Depends-On: openstack-k8s-operators/ci-framework#1892

Signed-off-by: Chandan Kumar <[email protected]>

softwarefactory-project-zuul bot commented Jun 19, 2024

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/09dfe40b25774c42b0b90b191da187ce

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 52m 40s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 18m 01s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 15m 01s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 9m 15s
✔️ cifmw-pod-pre-commit SUCCESS in 7m 32s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 54s
❌ ci-framework-openstack-meta-content-provider FAILURE in 49m 25s

raukadah force-pushed the meta_content_provider branch 2 times, most recently from 08d6468 to a3ac750 Compare

June 20, 2024 03:24

softwarefactory-project-zuul bot commented Jun 20, 2024

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/e7445e72b5674ccba28c882d8f1367d8

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 45m 19s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 17m 17s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 27m 25s
✔️ cifmw-data-plane-adoption-osp-17-to-extracted-crc SUCCESS in 2h 30m 11s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 10m 05s
❌ cifmw-pod-pre-commit FAILURE in 8m 28s
✔️ cifmw-pod-zuul-files SUCCESS in 5m 10s
✔️ ci-framework-openstack-meta-content-provider SUCCESS in 47m 41s
✔️ cifmw-molecule-edpm_prepare SUCCESS in 4m 58s

raukadah added a commit to raukadah/tcib that referenced this pull request


          [WIP] Testing meta content provider

0b9916b

Depends-On: https://review.opendev.org/c/openstack/nova/+/921687
Depends-On: openstack-k8s-operators/nova-operator#777
Depends-On: openstack-k8s-operators/ci-framework#1892

Signed-off-by: Chandan Kumar <[email protected]>

raukadah force-pushed the meta_content_provider branch from a3ac750 to 36f1651 Compare

June 21, 2024 03:06

raukadah added a commit to raukadah/tcib that referenced this pull request


          [WIP] Testing meta content provider

816e5e3

Depends-On: https://review.opendev.org/c/openstack/nova/+/921687
Depends-On: openstack-k8s-operators/nova-operator#777
Depends-On: openstack-k8s-operators/ci-framework#1892

Signed-off-by: Chandan Kumar <[email protected]>

raukadah added a commit to raukadah/tcib that referenced this pull request


          Replace tcib base with meta content provider

e25a4b5

openstack-k8s-operators/ci-framework#1892 adds
Meta content provider job to test opendev and github changes together.

This pr replaces tcib content provider with meta content provider
allowing users to test the content from different sources.

Signed-off-by: Chandan Kumar <[email protected]>

raukadah added a commit to raukadah/tcib that referenced this pull request


          Replace tcib base with meta content provider

7f283ff

openstack-k8s-operators/ci-framework#1892 adds
Meta content provider job to test opendev and github changes together.

This pr replaces tcib content provider with meta content provider
allowing users to test the content from different sources.

Signed-off-by: Chandan Kumar <[email protected]>

raukadah force-pushed the meta_content_provider branch from 876e8c4 to 06da6db Compare

June 24, 2024 12:12

github-actions bot added Ready For Review and removed Ready For Review labels

raukadah force-pushed the meta_content_provider branch from 06da6db to 7c52346 Compare

June 25, 2024 00:07

github-actions bot removed the Ready For Review label

raukadah added a commit to raukadah/tcib that referenced this pull request


          Replace tcib base with meta content provider

a0db69a

openstack-k8s-operators/ci-framework#1892 adds
Meta content provider job to test opendev and github changes together.

This pr replaces tcib content provider with meta content provider
allowing users to test the content from different sources.

Signed-off-by: Chandan Kumar <[email protected]>

github-actions bot added the Ready For Review label

marios reviewed

View reviewed changes

Contributor

marios left a comment

didn't spot anything major to complain about ;)
leaving approval for reviews call

zuul.d/content_provider.yaml Outdated

+                  name: openstack-meta-content-provider
+                  parent: openstack-k8s-operators-content-provider
+                  description: |
+                    A zuul job to build contents(rpms, openstack services containers,

Contributor

marios Jun 26, 2024

to build content (

ci/playbooks/meta_content_provider/meta_content_provider.yml Outdated

+                      ] | ansible.builtin.path_join
+                  }}
+              - name: "Run ci/playbooks/meta_content_provider/meta_content_provider.yml"

Contributor

marios Jun 26, 2024

remove quotes from task title for consistency?

raukadah force-pushed the meta_content_provider branch from 7c52346 to 67d3edf Compare

June 26, 2024 07:01

github-actions bot removed the Ready For Review label

raukadah added a commit to raukadah/tcib that referenced this pull request


          Replace tcib base with meta content provider

9df5ec2

openstack-k8s-operators/ci-framework#1892 adds
Meta content provider job to test opendev and github changes together.

This pr replaces tcib content provider with meta content provider
allowing users to test the content from different sources.

Signed-off-by: Chandan Kumar <[email protected]>

This was referenced Jun 26, 2024

DNM It's just test of using testproject together with operator changes openstack-k8s-operators/neutron-operator#371

Closed

DNM Just testing together change in the operator and u/s code openstack-k8s-operators/neutron-operator#372

Closed

github-actions bot added the Ready For Review label

lewisdenny reviewed

View reviewed changes

Collaborator

lewisdenny left a comment

Thank you Chandan, this will be a great feature to add to CI!

ci/playbooks/meta_content_provider/run.yml Outdated

+                    ansible.builtin.debug:
+                      msg: >-
+                        Running Content provider registry on
+                        {{ cifmw_rp_registry_ip | default('nowhere') }}

Collaborator

lewisdenny Jun 26, 2024

Could you please explain why we wouldn't fail when cifmw_rp_registry_ip is undefined?

Contributor Author

raukadah Jun 27, 2024

openstack-k8s-operators-content-provider is the parent of openstack-meta-content-provider. We run [Discover an IPv4 for provider job](Discover an IPv4 for provider job) in pre-run step to always set the value of cifmw_rp_registry_ip var . So this var is always going to set. It will be never undefined.

From a user point of view, this task does not provide any value. Let me drop that.

ci/playbooks/meta_content_provider/run.yml Outdated

+                      src: "{{ cifmw_artifacts_basedir }}/artifacts/ansible-vars.yml"
+                    register: _inner_ansible
+                  - name: Return Zuul Data

Collaborator

lewisdenny Jun 26, 2024

@cjeanner Is this a debug we want to keep?

zuul.d/content_provider.yaml Show resolved Hide resolved


          Add Meta content provider job to test opendev and github changes

7c09d46

Meta content provider does following things:
- It combines 3 content provider (operator, tcib and edpm image build)
  into one content provider to test all changes together.
- If the changes coming from opendev, tcib and os-net-config, then it
  will build the DLRN packages and create gating repo.
- If there is a gating and non os-net-config changes, then it will do
  the tcib build.
- If there is a edpm-image-build change, then it will perform the image
  build.
- All the built content will be pushed on single registry with different
  namespace.
- The content provider will expose the same zuul return vars to
  dependent job to pick proper built content.
- It also returns content_provider_os_registry_url var to dependent job
  and improve edpm_prepare role to set proper vars.
- Generate gating repo only when there are repos.

Note: It make changes to edpm_prepare to check edpm_image_build_output
key length. If no edpm image is built by meta content provider then
edpm_image_build_output will return an empty dict.

Signed-off-by: Chandan Kumar <[email protected]>

raukadah force-pushed the meta_content_provider branch from 67d3edf to 7c09d46 Compare

June 27, 2024 00:55

github-actions bot added Ready For Review and removed Ready For Review labels

lewisdenny approved these changes

View reviewed changes

Collaborator

lewisdenny left a comment

/approve

LGTM, quite a large change so let's keep an eye out once this merges.

openshift-ci bot assigned lewisdenny

openshift-ci bot added the lgtm label

Contributor

openshift-ci bot commented Jun 28, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lewisdenny

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [lewisdenny]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci bot added the approved label

github-actions bot added Ready For Review and removed Ready For Review labels

openshift-merge-bot bot merged commit 28db3e6 into main

5 checks passed

openshift-merge-bot bot deleted the meta_content_provider branch

June 28, 2024 06:20

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved lgtm Ready For Review