adds support in scripts for deploying nephio components on real K8S(non-kind) #269

vjayaramrh · 2024-06-18T15:44:16Z

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespaces from that line:

/kind bug
/kind cleanup
/kind design
/kind documentation
/kind failing-test

kind feature

/kind flake

What this PR does / why we need it:
This PR allows for deploying nephio management components on pre-existing K8S cluster using the existing scripts. On a machine with K8S installed, one would be able to execute the below command with correct values to install Nephio management components.

wget -O - https://raw.githubusercontent.com/nephio-project/test-infra/v2.0.0/e2e/provision/init.sh | sudo NEPHIO_DEBUG=false NEPHIO_BRANCH=v2.0.0 NEPHIO_USER=ubuntu K8S_CONTEXT=kubernetes-admin@kubernetes bash

The value for K8S_CONTEXT can be obtained by executing the below command
kubectl config get-contexts

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

vjayaramrh · 2024-06-19T02:26:14Z

/retest

liamfallon

/approve

vjayaramrh · 2024-06-20T15:25:55Z

/retest bootstrap-integration

nephio-prow · 2024-06-20T15:26:06Z

@vjayaramrh: The /retest command does not accept any targets.
The following commands are available to trigger required jobs:

/test bootstrap-integration
/test e2e-free5gc-ubuntu-focal
/test e2e-oai-ubuntu-jammy
/test images-hadolint
/test install-integration
/test kpt-integration
/test pre-submit-test-infra-validate-local
/test presubmit-test-infra-inrepoconfig-validation
/test provision-linter

The following commands are available to trigger optional jobs:

/test e2e-free5gc-fedora-34
/test e2e-oai-fedora-34

Use /test all to run the following jobs that were automatically triggered:

bootstrap-integration
e2e-free5gc-fedora-34
e2e-free5gc-ubuntu-focal
e2e-oai-fedora-34
e2e-oai-ubuntu-jammy
provision-linter

In response to this:

/retest bootstrap-integration

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

vjayaramrh · 2024-06-20T15:26:11Z

/retest provision-linter

nephio-prow · 2024-06-20T15:26:21Z

@vjayaramrh: The /retest command does not accept any targets.
The following commands are available to trigger required jobs:

/test bootstrap-integration
/test e2e-free5gc-ubuntu-focal
/test e2e-oai-ubuntu-jammy
/test images-hadolint
/test install-integration
/test kpt-integration
/test pre-submit-test-infra-validate-local
/test presubmit-test-infra-inrepoconfig-validation
/test provision-linter

The following commands are available to trigger optional jobs:

/test e2e-free5gc-fedora-34
/test e2e-oai-fedora-34

Use /test all to run the following jobs that were automatically triggered:

bootstrap-integration
e2e-free5gc-fedora-34
e2e-free5gc-ubuntu-focal
e2e-oai-fedora-34
e2e-oai-ubuntu-jammy
provision-linter

In response to this:

/retest provision-linter

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

vjayaramrh · 2024-06-20T15:28:18Z

/test provision-linter

vjayaramrh · 2024-06-20T15:29:25Z

/test bootstrap-integration

vjayaramrh · 2024-06-20T15:30:47Z

/test e2e-oai-fedora-34

vjayaramrh · 2024-06-20T15:37:29Z

@electrocucaracha I see the below in the bootstrap-integration, how do I fix this? THanks

TASK [Include bootstrap] ******************************************************* ERROR! 'REAL_K8S_TAG' is undefined. 'REAL_K8S_TAG' is undefined

electrocucaracha · 2024-06-20T15:33:30Z

e2e/provision/init.sh

-export ANSIBLE_CMD_EXTRA_VAR_LIST="nephio_catalog_repo_uri='$NEPHIO_CATALOG_REPO_URI'"
+K8S_CONTEXT=${K8S_CONTEXT:-"kind-kind"}
+K8S_VERSION=${K8S_VERSION:-"v1.29.2"}
+export ANSIBLE_CMD_EXTRA_VAR_LIST='{ "nephio_catalog_repo_uri": "'${NEPHIO_CATALOG_REPO_URI}'", "k8s": { "context" : "'${K8S_CONTEXT}'", "version" : "'$K8S_VERSION'" } }'


@vjayaramrh ANSIBLE_CMD_EXTRA_VAR_LIST variable can be provided by the user, so this overrides its value

Suggested change

export ANSIBLE_CMD_EXTRA_VAR_LIST='{ "nephio_catalog_repo_uri": "'${NEPHIO_CATALOG_REPO_URI}'", "k8s": { "context" : "'${K8S_CONTEXT}'", "version" : "'$K8S_VERSION'" } }'

export ANSIBLE_CMD_EXTRA_VAR_LIST="${ANSIBLE_CMD_EXTRA_VAR_LIST:-'{ "nephio_catalog_repo_uri": "'${NEPHIO_CATALOG_REPO_URI}'", "k8s": { "context" : "'${K8S_CONTEXT}'", "version" : "'$K8S_VERSION'" } }'}"

Good catch, thanks

The original code also did not account for user provided value for ANSIBLE_CMD_EXTRA_VAR_LIST. We should treat user provided input for this separately as we need to test out and validate other cases.
The good thing is I have tested out case where multiple --extra-vars can be supplied to a playbook and this can be used for future implementation.

This env variable is used by Vagrant, so overriding its value will result in a misfunctioning. Could you find a way to avoid that misfunctioning?

@electrocucaracha Thanks for the comment, I plan to open an issue to be fixed as part of a later PR. Would that be OK?. I noticed that you have 'https://github.com/nephio-project/nephio-catalog-packages.git in the link above that is not valid. Is that by design or should it have been https://github.com/nephio-project/catalog.git?

FYI I have opened an issue to address this post R3 at nephio-project/nephio#777

electrocucaracha · 2024-06-20T15:35:11Z

e2e/provision/init.sh

+if [ ${K8S_CONTEXT} == "kind-kind" ]; then
+    export ANSIBLE_TAG=all
+else
+    export ANSIBLE_TAG=real_k8s
+fi


Suggested change

if [ ${K8S_CONTEXT} == "kind-kind" ]; then

export ANSIBLE_TAG=all

else

export ANSIBLE_TAG=real_k8s

fi

ANSIBLE_TAG=all

if [ ${K8S_CONTEXT:-"kind-kind"} != "kind-kind" ]; then

ANSIBLE_TAG=production

fi

Good suggestion.
The value for ANSIBLE_TAG value will be kept as real_k8s for readability

electrocucaracha · 2024-06-20T15:36:59Z

e2e/provision/playbooks/cluster.yml

@@ -12,6 +12,7 @@
  hosts: all
  vars:
    container_engine: docker
+    tag_real_k8s: real_k8s


What about to using another name?

Suggested change

tag_real_k8s: real_k8s

k8s_type: production

Good suggestion, I like tag_k8s_type. I prefer value to be real_k8s as against production for the time being.

After thinking about it some more, decided to keep the original name as this is more like a constant and will always have the same value.

@liamfallon @radoslawc @arora-sagar any suggestion on this? My preference is to avoid the usage of real on these tags, but I'd like to hear other point of views.

@electrocucaracha what do you think about tag_nonkind_k8s: nonkind_k8s ?

@electrocucaracha I agree with you, even I don't like the term real and neither kind or non-kind because the same kind cluster we can make with minikube.

so how about k8s_distribution ?

I am thinking better name would be pre_deployed_k8s or similar instead of nonkind_k8s. The tag identifies the code sections that need to be executed on an existing cluster and skip the other parts(installing docker, kind and such). Will push a PR later with some of the feedback once this merges as we are close to code freeze and regularly failing e2e tests are not helping the cause.

electrocucaracha · 2024-06-20T15:44:51Z

@electrocucaracha I see the below in the bootstrap-integration, how do I fix this? THanks

TASK [Include bootstrap] ******************************************************* ERROR! 'REAL_K8S_TAG' is undefined. 'REAL_K8S_TAG' is undefined

That's because it is not defined on the molecule tests, btw could you use another lower case name, I was thinking in type, kind, environment, non-kind, non-existing

liamfallon · 2024-06-20T15:44:53Z

/retest

electrocucaracha · 2024-06-20T15:46:56Z

@liamfallon the bootstrap-integration failure is genuine, it has to be fixed prior to its merge

liamfallon · 2024-06-20T15:49:30Z

/retest

@liamfallon the bootstrap-integration failure is genuine, it has to be fixed prior to its merge

OK, thanks @electrocucaracha

…on-kind) Co-Authored-By: dkosteck <[email protected]>

vjayaramrh · 2024-06-26T02:20:46Z

/test e2e-free5gc-ubuntu-focal

vjayaramrh · 2024-06-26T03:51:06Z

/test e2e-oai-ubuntu-jammy

vjayaramrh · 2024-06-26T04:06:40Z

/test e2e-free5gc-fedora-34

vjayaramrh · 2024-06-26T04:39:48Z

/test e2e-oai-ubuntu-jammy

liamfallon · 2024-06-26T05:17:29Z

/test e2e-oai-ubuntu-jammy

liamfallon · 2024-06-26T06:12:46Z

"Deploy OAI Core Network Functions" on oai-amf failed in the e2e-oai-ubuntu-jimmy test, will retry:
/test e2e-oai-ubuntu-jammy

liamfallon · 2024-06-26T07:51:33Z

"Deploy OAI Core Network Functions" failed in the e2e-oai-ubuntu-jimmy test on "upf-edge" NF

liamfallon · 2024-06-26T07:53:09Z

/test e2e-oai-ubuntu-jammy

liamfallon · 2024-06-26T09:33:16Z

/test e2e-oai-ubuntu-jammy

vjayaramrh · 2024-06-26T13:21:16Z

/test e2e-oai-ubuntu-jammy

THis time it seems to have failed deploying oai-smf, I guess

vjayaramrh · 2024-06-26T13:21:31Z

/test e2e-oai-ubuntu-jammy

arora-sagar · 2024-06-26T13:27:25Z

@vjayaramrh Is it working locally for you?

vjayaramrh · 2024-06-26T13:29:59Z

@vjayaramrh Is it working locally for you?

I have not tried running the e2e locally, the same test passed CI earlier.

liamfallon · 2024-06-26T16:13:01Z

/test e2e-oai-ubuntu-jammy

vjayaramrh · 2024-06-26T17:01:23Z

@vjayaramrh Is it working locally for you?

@arora-sagar its now working locally

vjayaramrh · 2024-06-26T18:23:54Z

/test e2e-free5gc-fedora-34

nephio-prow · 2024-06-26T18:31:06Z

@vjayaramrh: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
e2e-free5gc-fedora-34	`8157350`	link	false	`/test e2e-free5gc-fedora-34`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

liamfallon · 2024-06-26T18:58:10Z

/approve
/lgtm

nephio-prow · 2024-06-26T18:58:15Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: liamfallon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [liamfallon]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

This is contingent on, and related to nephio-project/test-infra#269 so will be submitted as a draft for the time being. Documenting the additional variable of K8S_CONTEXT introduced in the linked PR for deploying Nephio components on non-kind clusters. Note also that in the "Real K8s Cluster" installation instructions the branch should be updated once nephio-project/test-infra#269 is merged and a new tag is available. Closes nephio-project/nephio#577 --------- Co-authored-by: Liam Fallon <[email protected]>

…on-kind) (nephio-project#269) Co-authored-by: dkosteck <[email protected]>

nephio-prow bot requested review from efiacor and liamfallon June 18, 2024 15:44

vjayaramrh marked this pull request as draft June 18, 2024 17:18

nephio-prow bot added the do-not-merge/work-in-progress label Jun 18, 2024

vjayaramrh force-pushed the non_kind_k8s branch from d94bad1 to 7ec9991 Compare June 18, 2024 19:31

vjayaramrh marked this pull request as ready for review June 18, 2024 19:53

nephio-prow bot removed the do-not-merge/work-in-progress label Jun 18, 2024

nephio-prow bot requested a review from electrocucaracha June 18, 2024 19:53

vjayaramrh force-pushed the non_kind_k8s branch 2 times, most recently from 9168063 to fdc04c1 Compare June 18, 2024 21:31

liamfallon reviewed Jun 19, 2024

View reviewed changes

vjayaramrh force-pushed the non_kind_k8s branch 2 times, most recently from 0f55cab to 1a1136a Compare June 19, 2024 12:42

electrocucaracha reviewed Jun 20, 2024

View reviewed changes

dkosteck mentioned this pull request Jun 20, 2024

Add K8S_CONTEXT label documentation nephio-project/docs#143

Merged

vjayaramrh force-pushed the non_kind_k8s branch from 1a1136a to 202b3da Compare June 21, 2024 03:40

nephio-prow bot added the approved label Jun 21, 2024

vjayaramrh force-pushed the non_kind_k8s branch 2 times, most recently from c0f7efb to ea891f9 Compare June 26, 2024 00:33

adds support in scripts for deploying nephio components on real K8S(n…

8157350

…on-kind) Co-Authored-By: dkosteck <[email protected]>

vjayaramrh force-pushed the non_kind_k8s branch from ea891f9 to 8157350 Compare June 26, 2024 01:01

vjayaramrh mentioned this pull request Jun 26, 2024

Evaluate and address code review feedback post R3 related to deploying Nephio management components on pre-installed k8s cluster nephio-project/nephio#777

Closed

nephio-prow bot assigned liamfallon Jun 26, 2024

nephio-prow bot added the lgtm label Jun 26, 2024

nephio-prow bot merged commit 2ccf753 into nephio-project:main Jun 26, 2024
6 of 8 checks passed

vjayaramrh deleted the non_kind_k8s branch July 18, 2024 20:33

dkosteck added a commit to vjayaramrh/test-infra that referenced this pull request Aug 15, 2024

adds support in scripts for deploying nephio components on real K8S(n…

ce14fde

…on-kind) (nephio-project#269) Co-authored-by: dkosteck <[email protected]>

	export ANSIBLE_CMD_EXTRA_VAR_LIST='{ "nephio_catalog_repo_uri": "'${NEPHIO_CATALOG_REPO_URI}'", "k8s": { "context" : "'${K8S_CONTEXT}'", "version" : "'$K8S_VERSION'" } }'
	export ANSIBLE_CMD_EXTRA_VAR_LIST="${ANSIBLE_CMD_EXTRA_VAR_LIST:-'{ "nephio_catalog_repo_uri": "'${NEPHIO_CATALOG_REPO_URI}'", "k8s": { "context" : "'${K8S_CONTEXT}'", "version" : "'$K8S_VERSION'" } }'}"

adds support in scripts for deploying nephio components on real K8S(non-kind) #269

adds support in scripts for deploying nephio components on real K8S(non-kind) #269

Conversation

vjayaramrh commented Jun 18, 2024

vjayaramrh commented Jun 19, 2024

liamfallon left a comment

Choose a reason for hiding this comment

vjayaramrh commented Jun 20, 2024

nephio-prow bot commented Jun 20, 2024

vjayaramrh commented Jun 20, 2024

nephio-prow bot commented Jun 20, 2024

vjayaramrh commented Jun 20, 2024

vjayaramrh commented Jun 20, 2024

vjayaramrh commented Jun 20, 2024

vjayaramrh commented Jun 20, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vjayaramrh Jun 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vjayaramrh Jun 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vjayaramrh Jun 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

electrocucaracha commented Jun 20, 2024

liamfallon commented Jun 20, 2024

electrocucaracha commented Jun 20, 2024

liamfallon commented Jun 20, 2024

vjayaramrh commented Jun 26, 2024

vjayaramrh commented Jun 26, 2024

vjayaramrh commented Jun 26, 2024

vjayaramrh commented Jun 26, 2024

liamfallon commented Jun 26, 2024

liamfallon commented Jun 26, 2024 • edited Loading

liamfallon commented Jun 26, 2024

liamfallon commented Jun 26, 2024

liamfallon commented Jun 26, 2024

vjayaramrh commented Jun 26, 2024

vjayaramrh commented Jun 26, 2024

arora-sagar commented Jun 26, 2024

vjayaramrh commented Jun 26, 2024

liamfallon commented Jun 26, 2024

vjayaramrh commented Jun 26, 2024

vjayaramrh commented Jun 26, 2024

nephio-prow bot commented Jun 26, 2024

liamfallon commented Jun 26, 2024

nephio-prow bot commented Jun 26, 2024

vjayaramrh Jun 21, 2024 •

edited

Loading

vjayaramrh Jun 20, 2024 •

edited

Loading

vjayaramrh Jun 21, 2024 •

edited

Loading

liamfallon commented Jun 26, 2024 •

edited

Loading