Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(oauth2proxy): enable self-signed TLS cert #210

Open
wants to merge 17 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions charts/cryostat/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,27 @@ cd cryostat-helm
helm install cryostat ./charts/cryostat
```

## Configuration

See the sections below for Helm chart values which can be used for configuring various aspects of the Cryostat installation.

If there are further customizations required to suit your deployment environment, choose the settings values that get
you closest to what you need, then manually edit the resulting Kubernetes objects to suit your requirements. You may
also consider using `helm install --dry-run` to render the Kubernetes YAML manifests without installing them, so that
you can apply your own customization patches as needed.

### TLS

When installed on OpenShift with `authentication.openshift.enabled=true`, the cluster's
["service serving certificates"](https://docs.openshift.com/container-platform/4.17/security/certificates/service-serving-certificate.html)
feature is used to enable managed TLS configuration on the exposed HTTP(S) ports.

When installed with `authentication.openshift.enabled=false` but `oauth2Proxy.tls.selfSigned.enabled=true` then a
self-signed TLS certificate will be generated at installation time to serve similar purposes. These TLS certificates
are not managed, will not automatically rotate, and will expire after 365 days. You will need to manually rotate the
certificates, or reinstall the chart, or else apply your own customizations to the Kubernetes manifests to automate TLS
certificate issuance and rotation.

## Parameters

### Cryostat Container
Expand Down Expand Up @@ -176,6 +197,7 @@ helm install cryostat ./charts/cryostat
| `oauth2Proxy.image.repository` | Repository for the OAuth2 Proxy container image | `quay.io/oauth2-proxy/oauth2-proxy` |
| `oauth2Proxy.image.pullPolicy` | Image pull policy for the OAuth2 Proxy container image | `Always` |
| `oauth2Proxy.image.tag` | Tag for the OAuth2 Proxy container image | `latest` |
| `oauth2Proxy.tls.selfSigned.enabled` | Whether a self-signed TLS certificate for oauth2-proxy HTTPS is generated and used. | `false` |
| `oauth2Proxy.resources.requests.cpu` | CPU resource request for the OAuth2 Proxy container. | `25m` |
| `oauth2Proxy.resources.requests.memory` | Memory resource request for the OAuth2 Proxy container. | `64Mi` |
| `oauth2Proxy.securityContext` | Security Context for the OAuth2 Proxy container. Defaults to meet "restricted" [Pod Security Standard](https://kubernetes.io/docs/concepts/security/pod-security-standards/#restricted). See: [SecurityContext](https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#security-context-1). | `{}` |
Expand Down
8 changes: 4 additions & 4 deletions charts/cryostat/templates/NOTES.txt
Original file line number Diff line number Diff line change
Expand Up @@ -34,12 +34,12 @@

{{- if $portForwards }}

{{ $listNum }}. Forward local port to the application's pod:
{{ $listNum }}. Forward local port to the application's service:
```
kubectl -n {{ .Release.Namespace }} wait --for=condition=available --timeout=60s deploy/{{ include "cryostat.deploymentName" . }}

export POD_NAME=$(kubectl get pods -n {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "cryostat.name" . }},app.kubernetes.io/instance={{ .Release.Name }},app.kubernetes.io/component=cryostat" --sort-by=.metadata.creationTimestamp -o jsonpath="{.items[-1:].metadata.name}")
kubectl -n {{ .Release.Namespace }} port-forward $POD_NAME 8080:http
export SVC_NAME=$(kubectl get services -n {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "cryostat.name" . }},app.kubernetes.io/instance={{ .Release.Name }},app.kubernetes.io/component=cryostat" --sort-by=.metadata.creationTimestamp -o jsonpath="{.items[-1:].metadata.name}")
kubectl -n {{ .Release.Namespace }} port-forward svc/$SVC_NAME {{ include "cryostat.core.service.port" . }}:{{ include "cryostat.core.service.scheme" . }}
```
{{- $listNum = add1 $listNum }}
{{- end }}
Expand All @@ -59,6 +59,6 @@
{{- else if contains "LoadBalancer" .Values.core.service.type }}
echo http://$SERVICE_IP:{{ .Values.core.service.httpPort }}
{{- else if contains "ClusterIP" .Values.core.service.type }}
http://localhost:8080
{{ include "cryostat.core.service.scheme" . }}://localhost:{{ include "cryostat.core.service.port" . }}
{{- end }}
```
21 changes: 21 additions & 0 deletions charts/cryostat/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,27 @@ Create the name of the service account to use.
{{- end }}
{{- end }}

{{/*
Cryostat service TLS enablement. Returns the string values "true" or "false".
*/}}
{{- define "cryostat.core.service.tls" -}}
{{ or .Values.authentication.openshift.enabled .Values.oauth2Proxy.tls.selfSigned.enabled }}
{{- end }}

{{/*
Cryostat service protocol. HTTPS if TLS is enabled, HTTP otherwise.
*/}}
{{- define "cryostat.core.service.scheme" -}}
{{ ternary "https" "http" ( include "cryostat.core.service.tls" . | eq "true" ) }}
{{- end }}

{{/*
Cryostat service port. 8443 if TLS is enabled, 8080 otherwise.
*/}}
{{- define "cryostat.core.service.port" -}}
{{ ternary 8443 8080 ( ( include "cryostat.core.service.scheme" . ) | eq "https" ) }}
{{- end }}

{{/*
Get or generate a default connection key for database.
*/}}
Expand Down
4 changes: 4 additions & 0 deletions charts/cryostat/templates/_oauth2Proxy.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -47,4 +47,8 @@ Create OAuth2 Proxy container. Configurations defined in alpha_config.yaml
mountPath: /etc/oauth2_proxy/basicauth
readOnly: true
{{- end }}
{{- if .Values.oauth2Proxy.tls.selfSigned.enabled }}
- name: {{ .Release.Name }}-oauth2proxy-tls
mountPath: /etc/tls/private
{{- end }}
{{- end}}
56 changes: 56 additions & 0 deletions charts/cryostat/templates/_reports_authproxy.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,62 @@
readOnly: true
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
{{- else if .Values.oauth2Proxy.tls.selfSigned.enabled }}
- name: {{ printf "%s-reports-%s" .Chart.Name "authproxy" }}
securityContext:
{{- toYaml (.Values.oauth2Proxy).securityContext | nindent 4 }}
image: "{{ (.Values.oauth2Proxy).image.repository }}:{{ (.Values.oauth2Proxy).image.tag }}"
imagePullPolicy: {{ (.Values.oauth2Proxy).image.pullPolicy }}
env:
- name: OAUTH2_PROXY_CLIENT_ID
value: dummy
- name: OAUTH2_PROXY_CLIENT_SECRET
value: none
- name: OAUTH2_PROXY_HTTP_ADDRESS
value: 0.0.0.0:4180
- name: OAUTH2_PROXY_HTTPS_ADDRESS
value: :8443
- name: OAUTH2_PROXY_TLS_CERT_FILE
value: /etc/tls/private/cert
- name: OAUTH2_PROXY_TLS_KEY_FILE
value: /etc/tls/private/key
- name: OAUTH2_PROXY_UPSTREAMS
value: http://localhost:10001/
- name: OAUTH2_PROXY_REDIRECT_URL
value: "http://localhost:4180/oauth2/callback"
- name: OAUTH2_PROXY_COOKIE_SECRET
valueFrom:
secretKeyRef:
name: {{ default (printf "%s-cookie-secret" .Release.Name) .Values.authentication.cookieSecretName }}
key: COOKIE_SECRET
optional: false
- name: OAUTH2_PROXY_EMAIL_DOMAINS
value: "*"
- name: OAUTH2_PROXY_HTPASSWD_USER_GROUP
value: write
- name: OAUTH2_PROXY_HTPASSWD_FILE
value: /etc/oauth2_proxy/basicauth/htpasswd
- name: OAUTH2_PROXY_SKIP_AUTH_ROUTES
value: "^/health$"
- name: OAUTH2_PROXY_PROXY_WEBSOCKETS
value: "false"
ports:
- containerPort: 4180
name: http
protocol: TCP
- containerPort: 8443
name: https
protocol: TCP
resources:
{{- toYaml .Values.oauth2Proxy.resources | nindent 4 }}
volumeMounts:
- name: {{ .Release.Name }}-reports-secret
mountPath: /etc/oauth2_proxy/basicauth
readOnly: true
{{- if .Values.oauth2Proxy.tls.selfSigned.enabled }}
- name: {{ .Release.Name }}-oauth2proxy-reports-tls
mountPath: /etc/tls/private
{{- end }}
{{- else }}
- name: {{ printf "%s-reports-%s" .Chart.Name "authproxy" }}
securityContext:
Expand Down
8 changes: 8 additions & 0 deletions charts/cryostat/templates/alpha_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,14 @@ data:
alpha_config.yaml: |-
server:
BindAddress: http://0.0.0.0:4180
{{- if .Values.oauth2Proxy.tls.selfSigned.enabled }}
SecureBindAddress: https://0.0.0.0:8443
TLS:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be wrapped in the if block above?

Key:
fromFile: /etc/tls/private/key
Cert:
fromFile: /etc/tls/private/cert
{{- end}}
upstreamConfig:
proxyRawPath: true
upstreams:
Expand Down
6 changes: 5 additions & 1 deletion charts/cryostat/templates/cryostat_deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ spec:
- name: QUARKUS_REST_CLIENT_EXTENSIONS_API_VERIFY_HOST
value: "false"
- name: QUARKUS_REST_CLIENT_REPORTS_URL
value: {{ printf "%s://cryostat:$(REPORTS_PASS_SECRET_KEY)@%s-reports.%s.svc:%d" (ternary "https" "http" (.Values.authentication.openshift).enabled) $fullName $.Release.Namespace (int .Values.reports.service.httpPort) }}
value: {{ printf "%s://cryostat:$(REPORTS_PASS_SECRET_KEY)@%s-reports.%s.svc:%d" (include "cryostat.core.service.scheme" .) $fullName $.Release.Namespace (int .Values.reports.service.httpPort) }}
{{- end }}
- name: QUARKUS_DATASOURCE_USERNAME
value: cryostat
Expand Down Expand Up @@ -215,4 +215,8 @@ spec:
- name: {{ .Release.Name }}-proxy-tls
secret:
secretName: {{ .Release.Name }}-proxy-tls
{{- else if .Values.oauth2Proxy.tls.selfSigned.enabled }}
- name: {{ .Release.Name }}-oauth2proxy-tls
secret:
secretName: {{ .Release.Name }}-oauth2proxy-tls
{{- end }}
2 changes: 1 addition & 1 deletion charts/cryostat/templates/cryostat_service.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ spec:
protocol: TCP
name: http
appProtocol: http
{{- if (.Values.authentication.openshift).enabled }}
{{- if ( include "cryostat.core.service.tls" . | eq "true" ) }}
- port: 443
targetPort: 8443
protocol: TCP
Expand Down
15 changes: 15 additions & 0 deletions charts/cryostat/templates/cryostat_tls_secret.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{{- if ( include "cryostat.core.service.tls" . | eq "true" ) }}
{{- $fullName := include "cryostat.fullname" . }}
{{- $cert := genSelfSignedCert $fullName nil nil 365 }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a question: This means the certificate expires after a year right? This means the users have to rotate the certificate themselves?

Any thoughts about depending on cert-manager to manage certificates like on the operator side?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, it'll expire after a year. Users have to rotate it themselves or figure out something else automated. I don't really want to get into adding an external dependency to the Helm chart, since part of the reason for the chart to exist is for users who can't use the Operator, ex. because they don't have full admin control of the cluster or whatever else. If the user needs TLS but can't install the Operator, then this gives them at least something to work with as a starting point, and then they can build whatever other automation they need on top to suit their particular deployment environment.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good! Thanks for explaining! I think a quick note in README about this would be helpful :D

apiVersion: v1
kind: Secret
metadata:
name: {{ .Release.Name }}-oauth2proxy-tls
labels:
{{- include "cryostat.labels" $ | nindent 4 }}
type: Opaque
immutable: true
data:
cert: {{ $cert.Cert | b64enc }}
key: {{ $cert.Key | b64enc }}
{{- end -}}
4 changes: 4 additions & 0 deletions charts/cryostat/templates/reports_deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -83,5 +83,9 @@ spec:
- name: {{ .Release.Name }}-proxy-tls
secret:
secretName: {{ .Release.Name }}-proxy-tls
{{- else if .Values.oauth2Proxy.tls.selfSigned.enabled }}
- name: {{ .Release.Name }}-oauth2proxy-reports-tls
secret:
secretName: {{ .Release.Name }}-oauth2proxy-reports-tls
{{- end }}
{{- end -}}
2 changes: 1 addition & 1 deletion charts/cryostat/templates/reports_service.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ spec:
type: {{ .Values.reports.service.type }}
ports:
- port: {{ .Values.reports.service.httpPort }}
targetPort: {{ ternary "https" "http" (.Values.authentication.openshift).enabled }}
targetPort: {{ include "cryostat.core.service.scheme" . }}
selector:
{{- include "cryostat.selectorLabels" $ | nindent 4 }}
app.kubernetes.io/component: reports
Expand Down
15 changes: 15 additions & 0 deletions charts/cryostat/templates/reports_tls_secret.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{{- if ( include "cryostat.core.service.tls" . | eq "true" ) }}
{{- $fullName := include "cryostat.fullname" . }}
{{- $cert := genSelfSignedCert (printf "%s-reports" $fullName) nil nil 365 }}
apiVersion: v1
kind: Secret
metadata:
name: {{ .Release.Name }}-oauth2proxy-reports-tls
labels:
{{- include "cryostat.labels" $ | nindent 4 }}
type: Opaque
immutable: true
data:
cert: {{ $cert.Cert | b64enc }}
key: {{ $cert.Key | b64enc }}
{{- end -}}
1 change: 1 addition & 0 deletions charts/cryostat/templates/tests/test-core-connection.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ metadata:
labels:
{{- include "cryostat.labels" . | nindent 4 }}
app.kubernetes.io/component: test-core-connection
charts.cryostat.io/role: helm-test
annotations:
"helm.sh/hook": test
spec:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ metadata:
labels:
{{- include "cryostat.labels" . | nindent 4 }}
app.kubernetes.io/component: test-grafana-connection
charts.cryostat.io/role: helm-test
annotations:
"helm.sh/hook": test
spec:
Expand Down
17 changes: 17 additions & 0 deletions charts/cryostat/tests/alpha_config_test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,23 @@ tests:
path: data['alpha_config.yaml']
pattern: "providers:\\s*- id: dummy\\s*name: Unused - Sign In Below\\s*clientId: CLIENT_ID\\s*clientSecret: CLIENT_SECRET\\s*provider: google"

- it: should create configmap with correct alpha configurations when self-signed TLS is enabled
set:
oauth2Proxy.tls.selfSigned.enabled: true
asserts:
- matchRegex:
path: data['alpha_config.yaml']
pattern: "server:\\s*BindAddress: http://0.0.0.0:4180\\s*SecureBindAddress: https://0.0.0.0:8443"
- matchRegex:
path: data['alpha_config.yaml']
pattern: "upstreamConfig:\\s*proxyRawPath: true\\s*upstreams:\\s*- id: cryostat\\s*path: /\\s*uri: http://localhost:8181"
- matchRegex:
path: data['alpha_config.yaml']
pattern: "- id: grafana\\s*path: /grafana/\\s*uri: http://localhost:3000"
- matchRegex:
path: data['alpha_config.yaml']
pattern: "providers:\\s*- id: dummy\\s*name: Unused - Sign In Below\\s*clientId: CLIENT_ID\\s*clientSecret: CLIENT_SECRET\\s*provider: google"

- it: should not create alpha_config when openshift authentication is enabled
set:
authentication:
Expand Down
19 changes: 19 additions & 0 deletions charts/cryostat/tests/cryostat_deployment_test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -416,11 +416,30 @@ tests:

- it: should validate volumes and volume mounts
asserts:
- lengthEqual:
count: 1
path: spec.template.spec.volumes
- exists:
path: spec.template.spec.volumes[?(@.name=='alpha-config')]
- exists:
path: spec.template.spec.containers[?(@.name=='cryostat-authproxy')].volumeMounts[?(@.mountPath=='/etc/oauth2_proxy/alpha_config')]

- it: should validate volumes and volume mounts with oauth2-proxy self-signed TLS enabled
set:
oauth2Proxy.tls.selfSigned.enabled: true
asserts:
- lengthEqual:
count: 2
path: spec.template.spec.volumes
- exists:
path: spec.template.spec.volumes[?(@.name=='alpha-config')]
- exists:
path: spec.template.spec.containers[?(@.name=='cryostat-authproxy')].volumeMounts[?(@.mountPath=='/etc/oauth2_proxy/alpha_config')]
- exists:
path: spec.template.spec.volumes[?(@.name=='RELEASE-NAME-oauth2proxy-tls')]
- exists:
path: spec.template.spec.containers[?(@.name=='cryostat-authproxy')].volumeMounts[?(@.mountPath=='/etc/tls/private')]

- it: should set image pull secrets if specified
set:
imagePullSecrets:
Expand Down
Loading