Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI][Windows] Support Windows tests with proxyAll enabled #2899

Merged
merged 1 commit into from
Oct 22, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,7 @@ Here are the trigger phrases for individual checks:
* `/test-windows-e2e`: Windows IPv4 e2e tests
* `/test-windows-conformance`: Windows IPv4 conformance tests
* `/test-windows-networkpolicy`: Windows IPv4 networkpolicy tests
* `/test-windows-proxyall-e2e`: Windows IPv4 e2e tests with proxyAll enabled
* `/test-ipv6-e2e`: Linux dual stack e2e tests
* `/test-ipv6-conformance`: Linux dual stack conformance tests
* `/test-ipv6-networkpolicy`: dLinux ual stack networkpolicy tests
Expand All @@ -168,6 +169,7 @@ Here are the trigger phrases for groups of checks:

* `/test-all`: Linux IPv4 tests
* `/test-windows-all`: Windows IPv4 tests
* `/test-windows-proxyall-all`: Windows IPv4 tests with proxyAll enabled
* `/test-ipv6-all`: Linux dual stack tests
* `/test-ipv6-only-all`: Linux IPv6 only tests

Expand Down
8 changes: 8 additions & 0 deletions ci/jenkins/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,14 @@ DOCKER_REGISTRY="$(head -n1 ci/docker-registry)"
./ci/jenkins/test.sh --testcase windows-networkpolicy --registry ${DOCKER_REGISTRY}
```

* windows e2e with proxyAll enabled: e2e tests in a Windows setup with proxyAll enabled.

```shell
#!/bin/bash
DOCKER_REGISTRY="$(head -n1 ci/docker-registry)"
./ci/jenkins/test.sh --testcase windows-e2e --registry ${DOCKER_REGISTRY} --proxyall
```

* [whole-conformance [daily]](https://jenkins.antrea-ci.rocks/job/antrea-whole-conformance-for-pull-request/):
community tests using sonobuoy, with certified-conformance mode.

Expand Down
25 changes: 22 additions & 3 deletions ci/jenkins/test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ MODE="report"
DOCKER_REGISTRY=$(head -n1 "${WORKSPACE}/ci/docker-registry")
GO_VERSION=$(head -n1 "${WORKSPACE}/build/images/deps/go-version")
IMAGE_PULL_POLICY="Always"
PROXY_ALL=false

WINDOWS_CONFORMANCE_FOCUS="\[sig-network\].+\[Conformance\]|\[sig-windows\]"
WINDOWS_CONFORMANCE_SKIP="\[LinuxOnly\]|\[Slow\]|\[Serial\]|\[Disruptive\]|\[Flaky\]|\[Feature:.+\]|\[sig-cli\]|\[sig-storage\]|\[sig-auth\]|\[sig-api-machinery\]|\[sig-apps\]|\[sig-node\]|\[Privileged\]|should be able to change the type from|\[sig-network\] Services should be able to create a functioning NodePort service \[Conformance\]|Service endpoints latency should not be very high|should be able to create a functioning NodePort service for Windows"
Expand All @@ -51,7 +52,8 @@ Run K8s e2e community tests (Conformance & Network Policy) or Antrea e2e tests o
--kubeconfig Path of cluster kubeconfig.
--workdir Home path for Go, vSphere information and antrea_logs during cluster setup. Default is $WORKDIR.
--testcase Windows install OVS, Conformance and Network Policy or Antrea e2e testcases on a Windows or Linux cluster.
--registry The docker registry to use instead of dockerhub."
--registry The docker registry to use instead of dockerhub.
--proxyall Enable proxyAll to test AntreaProxy."

function print_usage {
echoerr "$_usage"
Expand Down Expand Up @@ -82,6 +84,10 @@ case $key in
DOCKER_REGISTRY="$2"
shift 2
;;
--proxyall)
PROXY_ALL=true
shift
;;
-h|--help)
print_usage
exit 0
Expand Down Expand Up @@ -196,14 +202,18 @@ function collect_windows_network_info_and_logs {

function wait_for_antrea_windows_pods_ready {
kubectl apply -f "${WORKDIR}/antrea.yml"
kubectl apply -f "${WORKDIR}/kube-proxy-windows.yml"
if [[ "${PROXY_ALL}" == false ]]; then
kubectl apply -f "${WORKDIR}/kube-proxy-windows.yml"
fi
kubectl apply -f "${WORKDIR}/antrea-windows.yml"
kubectl rollout restart deployment/coredns -n kube-system
kubectl rollout status deployment/coredns -n kube-system
kubectl rollout status deployment.apps/antrea-controller -n kube-system
kubectl rollout status daemonset/antrea-agent -n kube-system
kubectl rollout status daemonset.apps/antrea-agent-windows -n kube-system
kubectl rollout status daemonset/kube-proxy-windows -n kube-system
if [[ "${PROXY_ALL}" == false ]]; then
kubectl rollout status daemonset/kube-proxy-windows -n kube-system
fi
kubectl get nodes -o wide --no-headers=true | awk -v role="$CONTROL_PLANE_NODE_ROLE" '$3 != role && $1 ~ /win/ {print $6}' | while read IP; do
for i in `seq 5`; do
sleep 5
Expand Down Expand Up @@ -270,6 +280,15 @@ function deliver_antrea_windows {
export KUBECONFIG=$KUBECONFIG_PATH
export_govc_env_var

if [[ "${PROXY_ALL}" == true ]]; then
echo "====== Updating yaml files to enable proxyAll ======"
KUBERNETES_SVC_EP_IP=$(kubectl get endpoints kubernetes -o jsonpath='{.subsets[0].addresses[0].ip}')
KUBERNETES_SVC_EP_PORT=$(kubectl get endpoints kubernetes -o jsonpath='{.subsets[0].ports[0].port}')
KUBERNETES_SVC_EP_ADDR="${KUBERNETES_SVC_EP_IP}:${KUBERNETES_SVC_EP_PORT}"
sed -i "s|#kubeAPIServerOverride: \"\"|kubeAPIServerOverride: \"${KUBERNETES_SVC_EP_ADDR}\"|g" build/yamls/antrea.yml build/yamls/antrea-windows.yml
sed -i "s|#proxyAll: false|proxyAll: true|g" build/yamls/antrea.yml build/yamls/antrea-windows.yml
fi

cp -f build/yamls/*.yml $WORKDIR
docker save -o antrea-ubuntu.tar projects.registry.vmware.com/antrea/antrea-ubuntu:latest

Expand Down