CASMPET-4080: Updates for Istio 1.7.8

The charts are updated for Istio 1.7.8. Here's a summary of the
changes.

* cray-istio-operator

I updated the charts/istio-operator subchart from the 1.7.8
distribution.

I got rid of the wait-jobs hook because this isn't necessary since Helm
takes care of making sure the CRD is ready.

On a related note, I found that when I upgraded from Istio 1.6.13 that
Helm deleted the IstioOperator CRD. To work around this, there's an
upgrade hook that recreates the IstioOperator CRD if it doens't exist.

* cray-istio-deploy

Just changed the default image tags and removed things from the README
that weren't accurate.

* cray-istio

I updated the charts/istio and charts/ingressgatewayhmn subcharts with
the latest version from the 1.7.8 distribution.

I removed the transfer encoding workaround ( CASMPET-3079 ). The
upstream bug is fixed: envoyproxy/envoy#10041

I removed the tcp-stats-filter-1.6 memory leak workaround ( CASMPET-4026 ).
Ths upstream bug is fixed: istio/istio#24720

The istio-ingressgateway now needs `runAsRoot: true`, see
https://istio.io/latest/news/releases/1.7.x/announcing-1.7/upgrade-notes/#gateways-run-as-non-root .

kubernetes/cray-istio-deploy/README.md

@@ -1,17 +1,3 @@
 
 This chart creates the IstioOperator object for the Cray system.
 It will be run after the cray-istio-operator chart starts the Istio Operator which defines the IstioOperator custom resource.
-
-Istio is currently configured using the Helm passthrough method, see the info box on
-https://archive.istio.io/v1.5/docs/setup/install/istioctl/#customizing-the-configuration .
-The Helm passthrough method is used because we used Helm previously and this
-makes for minimal changes to the config.
-
-Note that this chart is configured to disable the ingress gateway component.
-This is because Istio 1.5 has a bug/limitation where it always deploys a Gateway
-but it doesn't provide the options that we're already using for that Gateway.
-The ingress-gateway is deployed by the cray-istio chart which runs after this
-chart.
-See https://github.com/istio/istio/issues/21577 for a discussion of the bug.
-According to this issue the Gateway doesn't get created in 1.6. If that's the
-case we can change this chart to enable the ingress-gateway component.

kubernetes/cray-istio-deploy/values.yaml

@@ -1,6 +1,6 @@
 
 hub: dtr.dev.cray.com/cray
-tag: 1.6.13-cray1-20210202183713_549528c5da
+tag: 1.7.8-cray1
 
 kubectl:
   image:
@@ -9,7 +9,7 @@ kubectl:
 
 pilot:
   hub: dtr.dev.cray.com/cray
-  tag: 1.6.13-cray1-20210202183713_549528c5da
+  tag: 1.7.8-cray1
 
 meshConfig:
   accessLogFile: /dev/stdout

kubernetes/cray-istio-operator/Chart.yaml

@@ -1,5 +1,5 @@
 apiVersion: v1
-appVersion: 1.6.13
+appVersion: 1.7.8
 name: cray-istio-operator
 description: Deploys the istio operator for Cray systems.
 version: 1.18.3

kubernetes/cray-istio-operator/README.md

@@ -1,8 +1,17 @@
 
 This deploys the Istio operator. There are instructions here:
-https://archive.istio.io/v1.5/docs/setup/install/standalone-operator/
+https://istio.io/v1.7/docs/setup/install/operator/
 
 The istio-operator chart in the charts/ directory was copied from the istio
 release which is available for download at
 https://github.com/istio/istio/releases/ .
-The chart is in `install/kubernetes/operator/charts/istio-operator`.
+The chart is in `manifests/charts/istio-operator`.
+
+# Recreate IstioOperator CRD on upgrade
+
+When upgrading from the 1.6 istio-operator chart to 1.7 Helm winds up deleting
+the IstioOperator CRD. The workaround is to have a post-upgrade hook that
+checks for the condition and runs kubectl apply to recreate the
+IstioOperator CRD.
+
+This can probably be removed in the next Istio upgrade.

kubernetes/cray-istio-operator/charts/istio-operator/Chart.yaml

@@ -1,12 +1,12 @@
 apiVersion: v1
 name: istio-operator
-version: 1.6.0
+version: 1.7.0
 tillerVersion: ">=2.7.2"
 description: Helm chart for deploying Istio operator
 keywords:
   - istio
   - operator
 sources:
-  - http://github.com/istio/istio/operator
+  - https://github.com/istio/istio/tree/master/operator
 engine: gotpl
-icon: https://istio.io/favicons/android-192x192.png
+icon: https://istio.io/latest/favicons/android-192x192.png

kubernetes/cray-istio-operator/charts/istio-operator/crds/crd-operator.yaml

@@ -0,0 +1,74 @@
+# SYNC WITH manifests/charts/base/files
+apiVersion: apiextensions.k8s.io/v1
+kind: CustomResourceDefinition
+metadata:
+  name: istiooperators.install.istio.io
+  labels:
+    release: istio
+spec:
+  group: install.istio.io
+  names:
+    kind: IstioOperator
+    plural: istiooperators
+    singular: istiooperator
+    shortNames:
+      - iop
+  scope: Namespaced
+  versions:
+    - additionalPrinterColumns:
+        - description: Istio control plane revision
+          jsonPath: .spec.revision
+          name: Revision
+          type: string
+        - description: IOP current state
+          jsonPath: .status.status
+          type: string
+          name: Status
+        - jsonPath: .metadata.creationTimestamp
+          description:
+            "CreationTimestamp is a timestamp representing the server time when
+            this object was created. It is not guaranteed to be set in happens-before order
+            across separate operations. Clients may not set this value. It is represented
+            in RFC3339 form and is in UTC. Populated by the system. Read-only. Null for
+            lists. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#metadata"
+          name: Age
+          type: date
+      name: v1alpha1
+      schema:
+        openAPIV3Schema:
+          properties:
+            apiVersion:
+              description:
+                "APIVersion defines the versioned schema of this representation
+                of an object. Servers should convert recognized schemas to the latest
+                internal value, and may reject unrecognized values.
+                More info: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#resources"
+              type: string
+            kind:
+              description:
+                "Kind is a string value representing the REST resource this
+                object represents. Servers may infer this from the endpoint the client
+                submits requests to. Cannot be updated. In CamelCase.
+                More info: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#types-kinds"
+              type: string
+            spec:
+              description:
+                "Specification of the desired state of the istio control plane resource.
+                More info: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#spec-and-status"
+              x-kubernetes-preserve-unknown-fields: true
+              type: object
+            status:
+              description:
+                "Status describes each of istio control plane component status at the current time.
+                0 means NONE, 1 means UPDATING, 2 means HEALTHY, 3 means ERROR, 4 means RECONCILING.
+                More info: https://github.com/istio/api/blob/master/operator/v1alpha1/istio.operator.v1alpha1.pb.html &
+                https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#spec-and-status"
+              x-kubernetes-preserve-unknown-fields: true
+              type: object
+          type: object
+      served: true
+      storage: true
+      subresources:
+        status: {}
+---
+

kubernetes/cray-istio-operator/charts/istio-operator/templates/clusterrole.yaml

@@ -2,7 +2,7 @@ apiVersion: rbac.authorization.k8s.io/v1
 kind: ClusterRole
 metadata:
   creationTimestamp: null
-  name: istio-operator
+  name: istio-operator{{- if not (eq .Values.revision "") }}-{{ .Values.revision }}{{- end }}
 rules:
 # istio groups
 - apiGroups:
@@ -29,12 +29,6 @@ rules:
   - '*'
   verbs:
   - '*'
-- apiGroups:
-  - rbac.istio.io
-  resources:
-  - '*'
-  verbs:
-  - '*'
 - apiGroups:
   - security.istio.io
   resources:
@@ -81,6 +75,7 @@ rules:
   verbs:
   - get
   - create
+  - update
 - apiGroups:
   - policy
   resources:
@@ -96,6 +91,14 @@ rules:
   - rolebindings
   verbs:
   - '*'
+- apiGroups:
+  - coordination.k8s.io
+  resources:
+  - leases
+  verbs:
+  - get
+  - create
+  - update
 - apiGroups:
   - ""
   resources:
@@ -104,6 +107,7 @@ rules:
   - events
   - namespaces
   - pods
+  - pods/proxy
   - persistentvolumeclaims
   - secrets
   - services

kubernetes/cray-istio-operator/charts/istio-operator/templates/clusterrole_binding.yaml

@@ -1,13 +1,13 @@
 kind: ClusterRoleBinding
 apiVersion: rbac.authorization.k8s.io/v1
 metadata:
-  name: istio-operator
+  name: istio-operator{{- if not (eq .Values.revision "") }}-{{ .Values.revision }}{{- end }}
 subjects:
 - kind: ServiceAccount
-  name: istio-operator
+  name: istio-operator{{- if not (eq .Values.revision "") }}-{{ .Values.revision }}{{- end }}
   namespace: {{.Values.operatorNamespace}}
 roleRef:
   kind: ClusterRole
-  name: istio-operator
+  name: istio-operator{{- if not (eq .Values.revision "") }}-{{ .Values.revision }}{{- end }}
   apiGroup: rbac.authorization.k8s.io
 ---

kubernetes/cray-istio-operator/charts/istio-operator/templates/crds.yaml

@@ -0,0 +1,6 @@
+{{- if .Values.enableCRDTemplates -}}
+{{- range $path, $bytes := .Files.Glob "crds/*.yaml" -}}
+---
+{{ $.Files.Get $path }}
+{{- end -}}
+{{- end -}}

kubernetes/cray-istio-operator/charts/istio-operator/templates/deployment.yaml

@@ -2,7 +2,7 @@ apiVersion: apps/v1
 kind: Deployment
 metadata:
   namespace: {{.Values.operatorNamespace}}
-  name: istio-operator
+  name: istio-operator{{- if not (eq .Values.revision "") }}-{{ .Values.revision }}{{- end }}
 spec:
   replicas: 1
   selector:
@@ -13,30 +13,39 @@ spec:
       labels:
         name: istio-operator
     spec:
-      serviceAccountName: istio-operator
+      serviceAccountName: istio-operator{{- if not (eq .Values.revision "") }}-{{ .Values.revision }}{{- end }}
       containers:
         - name: istio-operator
           image: {{.Values.hub}}/operator:{{.Values.tag}}
           command:
           - operator
           - server
+          securityContext:
+            allowPrivilegeEscalation: false
+            capabilities:
+              drop:
+              - ALL
+            privileged: false
+            readOnlyRootFilesystem: true
+            runAsGroup: 1337
+            runAsUser: 1337
+            runAsNonRoot: true
           imagePullPolicy: IfNotPresent
           resources:
-            limits:
-              cpu: 200m
-              memory: 256Mi
-            requests:
-              cpu: 50m
-              memory: 128Mi
+{{ toYaml .Values.operator.resources | trim | indent 12 }}
           env:
             - name: WATCH_NAMESPACE
-              value: {{.Values.istioNamespace}}
+              value: {{.Values.watchedNamespaces | quote}}
             - name: LEADER_ELECTION_NAMESPACE
-              value: {{.Values.operatorNamespace}}
+              value: {{.Values.operatorNamespace | quote}}
             - name: POD_NAME
               valueFrom:
                 fieldRef:
                   fieldPath: metadata.name
             - name: OPERATOR_NAME
-              value: {{.Values.operatorNamespace}}
+              value: {{.Values.operatorNamespace | quote}}
+            - name: WAIT_FOR_RESOURCES_TIMEOUT
+              value: {{.Values.waitForResourcesTimeout | quote}}
+            - name: REVISION
+              value: {{.Values.revision | quote}}
 ---

kubernetes/cray-istio-operator/charts/istio-operator/templates/service.yaml

@@ -4,7 +4,7 @@ metadata:
   namespace: {{.Values.operatorNamespace}}
   labels:
     name: istio-operator
-  name: istio-operator
+  name: istio-operator{{- if not (eq .Values.revision "") }}-{{ .Values.revision }}{{- end }}
 spec:
   ports:
   - name: http-metrics

kubernetes/cray-istio-operator/charts/istio-operator/templates/service_account.yaml

@@ -2,5 +2,5 @@ apiVersion: v1
 kind: ServiceAccount
 metadata:
   namespace: {{.Values.operatorNamespace}}
-  name: istio-operator
+  name: istio-operator{{- if not (eq .Values.revision "") }}-{{ .Values.revision }}{{- end }}
 ---

kubernetes/cray-istio-operator/charts/istio-operator/values.yaml

@@ -1,4 +1,25 @@
 hub: gcr.io/istio-testing
-tag: 1.6-dev
+tag: latest
+
 operatorNamespace: istio-operator
-istioNamespace: istio-system
+
+# Used to replace istioNamespace to support operator watch multiple namespaces.
+watchedNamespaces: istio-system
+waitForResourcesTimeout: 300s
+
+# Used for helm2 to add the CRDs to templates.
+enableCRDTemplates: false
+
+# revision for the operator resources
+revision: ""
+
+# Operator resource defaults
+operator:
+  resources:
+    limits:
+      cpu: 200m
+      memory: 256Mi
+    requests:
+      cpu: 50m
+      memory: 128Mi
+