Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add check-cluster-up script for KinD providers #645

Merged
merged 1 commit into from
Oct 20, 2021
Merged

Add check-cluster-up script for KinD providers #645

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 88 additions & 0 deletions cluster-up/cluster/kind-1.19-sriov/conformance.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
{
"Description": "DEFAULT",
"UUID": "c3bc7d76-6ce8-4c8a-8bcb-5c7ae5fb22a3",
"Version": "v0.50.0",
"ResultsDir": "/tmp/sonobuoy",
"Resources": [
"apiservices",
"certificatesigningrequests",
"clusterrolebindings",
"clusterroles",
"componentstatuses",
"configmaps",
"controllerrevisions",
"cronjobs",
"customresourcedefinitions",
"daemonsets",
"deployments",
"endpoints",
"ingresses",
"jobs",
"leases",
"limitranges",
"mutatingwebhookconfigurations",
"namespaces",
"networkpolicies",
"nodes",
"persistentvolumeclaims",
"persistentvolumes",
"poddisruptionbudgets",
"pods",
"podlogs",
"podsecuritypolicies",
"podtemplates",
"priorityclasses",
"replicasets",
"replicationcontrollers",
"resourcequotas",
"rolebindings",
"roles",
"servergroups",
"serverversion",
"serviceaccounts",
"services",
"statefulsets",
"storageclasses",
"validatingwebhookconfigurations",
"volumeattachments"
],
"Filters": {
"Namespaces": ".*",
"LabelSelector": ""
},
"Limits": {
"PodLogs": {
"Namespaces": "",
"SonobuoyNamespace": true,
"FieldSelectors": [],
"LabelSelector": "",
"Previous": false,
"SinceSeconds": null,
"SinceTime": null,
"Timestamps": false,
"TailLines": null,
"LimitBytes": null,
"LimitSize": "",
"LimitTime": ""
}
},
"QPS": 30,
"Burst": 50,
"Server": {
"bindaddress": "0.0.0.0",
"bindport": 8080,
"advertiseaddress": "",
"timeoutseconds": 10800
},
"Plugins": null,
"PluginSearchPath": [
"./plugins.d",
"/etc/sonobuoy/plugins.d",
"~/sonobuoy/plugins.d"
],
"Namespace": "sonobuoy",
"WorkerImage": "projects.registry.vmware.com/sonobuoy/sonobuoy:v0.50.0",
"ImagePullPolicy": "IfNotPresent",
"ImagePullSecrets": "",
"ProgressUpdatesPort": "8099"
}
77 changes: 77 additions & 0 deletions cluster-up/cluster/kind/check-cluster-up.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
#!/bin/bash
#
# This file is part of the KubeVirt project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Copyright 2021 Red Hat, Inc.

set -exuo pipefail

SCRIPT_PATH=$(dirname "$(realpath "$0")")

kubevirtci_path="$(realpath "${SCRIPT_PATH}/../../..")/"
PROVIDER_PATH="${kubevirtci_path}/cluster-up/cluster/${KUBEVIRT_PROVIDER}"

RUN_KUBEVIRT_CONFORMANCE=${RUN_KUBEVIRT_CONFORMANCE:-"false"}

(
cd $kubevirtci_path
kubectl="./cluster-up/kubectl.sh"
echo "Wait for pods to be ready.."
timeout 5m bash -c "until ${kubectl} wait --for=condition=Ready pod --timeout=30s --all -A; do sleep 1; done"
timeout 5m bash -c "until ${kubectl} wait --for=condition=Ready pod --timeout=30s -n kube-system --all; do sleep 1; done"
${kubectl} get nodes
${kubectl} get pods -A
echo ""

nodes=$(${kubectl} get nodes --no-headers | awk '{print $1}')
for node in $nodes; do
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some thing nice is to create an array of command strings and then iterate on it , executing it,
wdyt?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree its nicer, I didnt do it because I wanted to keep the log messages before each command.
Putting it in bash equivalent for map doesn't make things prettier in this case.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can still keep log messages or whatever you desire in the loop isnt it ?
something like this wouldn't be nicer?

declare -a arr=("element1" "element2" "element3")
for i in "${arr[@]}"
do
   echo "$i" # exec here instead
done

Copy link
Contributor Author

@ormergi ormergi Aug 11, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can still keep log messages or whatever you desire in the loop isnt it ?

It will print the command we executes and not its descriptions, for example
for ip a command, it will print:
[sriov-worker] ip r
instead of:
[sriov-worker] network interfaces status:

It makes the log easier to follow, cosmetics.

node_exec="docker exec ${node}"
echo "[$node] network interfaces status:"
${node_exec} ip a
echo ""
echo "[$node] route table:"
${node_exec} ip r
echo ""
echo "[$node] hosts file:"
${node_exec} cat /etc/hosts
echo ""
echo "[$node] resolve config:"
${node_exec} cat /etc/resolv.conf
echo ""
done

if [ "$RUN_KUBEVIRT_CONFORMANCE" == "true" ]; then
nightly_build_base_url="https://storage.googleapis.com/kubevirt-prow/devel/nightly/release/kubevirt/kubevirt"
latest=$(curl -sL "${nightly_build_base_url}/latest")

echo "Deploy latest nighly build Kubevirt"
if [ "$(kubectl get kubevirts -n kubevirt kubevirt -ojsonpath='{.status.phase}')" != "Deployed" ]; then
${kubectl} apply -f "${nightly_build_base_url}/${latest}/kubevirt-operator.yaml"
${kubectl} apply -f "${nightly_build_base_url}/${latest}/kubevirt-cr.yaml"
fi
${kubectl} wait -n kubevirt kv kubevirt --for condition=Available --timeout 15m
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe should be part of the if, else it means that the kubevirt is deployed but here you are checking that its also Available, again up to you

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check is not part of the if in order to prevent running tests in case not all kubevirt components are ready.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe 15 minutes is too much ?
how much other repos use

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on hack/cluster-deploy.sh it check every 5 minutes for 25 minutes
https://github.com/kubevirt/kubevirt/blob/371958b07a340d0ba9e9d1e9a2936411e395a5df/hack/cluster-deploy.sh#L154
From what I saw sometime it takes 2-3 rounds to be ready, rarely more then 10 minutes , so I picked 15 minutes timeout.


echo "Run latest nighly build Kubevirt conformance tests"
kubevirt_plugin="--plugin ${nightly_build_base_url}/${latest}/conformance.yaml"
SONOBUOY_EXTRA_ARGS="${SONOBUOY_EXTRA_ARGS} ${kubevirt_plugin}"

commit=$(curl -sL "${nightly_build_base_url}/${latest}/commit")
commit="${commit:0:9}"
container_tag="--plugin-env kubevirt-conformance.CONTAINER_TAG=${latest}_${commit}"
SONOBUOY_EXTRA_ARGS="${SONOBUOY_EXTRA_ARGS} ${container_tag}"

hack/conformance.sh ${PROVIDER_PATH}/conformance.json
fi
)