Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document RBAC set up for ingress key cert protection. #755

Merged
merged 9 commits into from
Feb 6, 2018
198 changes: 198 additions & 0 deletions _docs/tasks/traffic-management/ingress.md
Original file line number Diff line number Diff line change
Expand Up @@ -269,6 +269,204 @@ rules.
curl -I -k https://$INGRESS_HOST/status/200
```

1. Configuring RBAC for ingress key/cert.

There are service accounts which can access this ingress key/cert, and this leads to risks of
leaking key/cert. We can set up Role-Based Access Control ("RBAC") to protect it.
[istio/install/kubernetes/istio.yaml](https://github.com/istio/istio/blob/master/install/kubernetes/istio.yaml)
defines ClusterRoles and ClusterRoleBindings which allow service accounts in namespace istio-system
to access all secret resources. We need to update or replace these RBAC set up to only allow
istio-ingress-service-account to access ingress key/cert.

We can use kubectl to list all secrets in namespace istio-system that we need to protect using RBAC.

```bash
kubectl get secrets -n istio-system
```
This produces the following output:

```bash
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have to display everything, please focus on ingress related one.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Thanks!

NAME TYPE DATA AGE
istio-ca-secret istio.io/ca-root 2 7d
istio-ingress-certs kubernetes.io/tls 2 7d
istio-ingress-service-account-token-kwr85 kubernetes.io/service-account-token 3 7d
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove this one as well? And istio-ca-secret?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

istio.istio-ingress-service-account istio.io/key-and-cert 3 7d
......
```

1. Update RBAC set up for istio-pilot-service-account and istio-mixer-istio-service-account.

Delete existing ClusterRoleBindings and ClusterRole.

```bash
kubectl delete ClusterRoleBinding istio-pilot-admin-role-binding-istio-system
kubectl delete ClusterRoleBinding istio-mixer-admin-role-binding-istio-system
kubectl delete ClusterRole istio-mixer-istio-system
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a similar role for istio-pilot-istio-system?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any way to recover the deletion here at the end of this file? The idea is to clean up the change in this file after everything is done.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And what privileges will pilot/mixer lose after this change? I am afraid this will break existing system. Let me add some pilot and mixer guy.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

istio-pilot-istio-system is also bound to account istio-ingress-service-account by istio-ingress-admin-role-binding-istio-system. We will delete istio-pilot-istio-system in next step, after deleting that ClusterRoleBinding object.

According to https://github.com/istio/istio/blob/11dc2fb937a23157215fffa06921cf507d27f757/install/kubernetes/istio.yaml#L22
istio-pilot-istio-system grants access permission to many resources. We need to bind those rules to istio-pilot-service-account. We should apply similar role binding process to account istio-mixer-service-account and istio-ca-service-account

```

Create istio-pilot-mixer-istio-system.yaml, which allows istio-pilot-service-account and
istio-mixer-service-account to read all kubernetes.io/service-account-token,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think anyone needs to read kubernetes.io/service-account-token, no?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree. That line is removed.

istio.io/key-and-cert, and istio.io/ca-root types of secret instances.

```bash
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: istio-pilot-mixer-istio-system
rules:
- apiGroups: [""] # "" indicates the core API group
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here and below. Please avoid showing unrelated content.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

resources: ["secrets"]
resourceNames: ["istio-ca-service-account-token-rl4xm"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems quite dangerous to hard-code names with randomized tokens (rl4xm).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a kubernetes.io/service-account-token, and it is removed.

verbs: ["get", "list", "watch"]
- apiGroups: [""] # "" indicates the core API group
resources: ["secrets"]
resourceNames: ["istio.istio-ca-service-account"]
verbs: ["get", "list", "watch"]
- apiGroups: [""] # "" indicates the core API group
resources: ["secrets"]
resourceNames: ["istio-ca-secret"]
verbs: ["get", "list", "watch"]
......
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: istio-pilot-mixer-admin-role-binding-istio-system
subjects:
- kind: ServiceAccount
name: istio-pilot-service-account
namespace: istio-system
- kind: ServiceAccount
name: istio-mixer-service-account
namespace: istio-system
Copy link
Member

@lookuptable lookuptable Dec 6, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we're binding the same role to both mixer SA and pilot SA. Is this desirable? IIRC mixer and pilot need different permissions.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, they need different permissions. I have create separate ClusterRoles for each of them.

roleRef:
kind: ClusterRole
name: istio-pilot-mixer-istio-system
apiGroup: rbac.authorization.k8s.io
```

```bash
kubectl apply -f istio-pilot-mixer-istio-system.yaml
```

1. Update RBAC set up for istio-ingress-service-account.

Delete existing ClusterRoleBinding and ClusterRole.

```bash
kubectl delete clusterrolebinding istio-ingress-admin-role-binding-istio-system
kubectl delete ClusterRole istio-pilot-istio-system
```

Create istio-ingress-istio-system.yaml which allows istio-ingress-service-account to read
istio-ingress-certs as well as other secret instances.

```bash
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: istio-ingress-istio-system
rules:
- apiGroups: [""] # "" indicates the core API group
resources: ["secrets"]
resourceNames: ["istio-ca-service-account-token-rl4xm"]
verbs: ["get", "list", "watch"]
- apiGroups: [""] # "" indicates the core API group
resources: ["secrets"]
resourceNames: ["istio.istio-ca-service-account"]
verbs: ["get", "list", "watch"]
- apiGroups: [""] # "" indicates the core API group
resources: ["secrets"]
resourceNames: ["istio-ca-secret"]
verbs: ["get", "list", "watch"]
......
- apiGroups: [""] # "" indicates the core API group
resources: ["secrets"]
resourceNames: ["istio-ingress-certs"]
verbs: ["get", "list", "watch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: istio-ingress-admin-role-binding-istio-system
subjects:
- kind: ServiceAccount
name: istio-ingress-service-account
namespace: istio-system
roleRef:
kind: ClusterRole
name: istio-ingress-istio-system
apiGroup: rbac.authorization.k8s.io
```

```bash
kubectl apply -f istio-ingress-istio-system.yaml
```

1. Update RBAC set up for istio-ca-service-account.

Create istio-ca-istio-system.yaml, which updates existing ClusterRole istio-ca-istio-system that
allows istio-ca-service-account to read, create and modify all istio.io/key-and-cert,
kubernetes.io/service-account-token and istio.io/ca-root types of secrets.

```bash
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: istio-ca-istio-system
rules:
- apiGroups: [""] # "" indicates the core API group
resources: ["secrets"]
resourceNames: ["istio-ca-service-account-token-rl4xm"]
verbs: ["get", "list", "watch", "create", "update"]
- apiGroups: [""] # "" indicates the core API group
resources: ["secrets"]
resourceNames: ["istio.istio-ca-service-account"]
verbs: ["get", "list", "watch", "create", "update"]
- apiGroups: [""] # "" indicates the core API group
resources: ["secrets"]
resourceNames: ["istio-ca-secret"]
verbs: ["get", "list", "watch", "create", "update"]
......
```

```bash
kubectl apply -f istio-ca-istio-system.yaml
```
1. Verify that the new ClusterRoles work as expected.

```bash
kubectl auth can-i get secret/istio-ingress-certs --as system:serviceaccount:istio-system:istio-ingress-service-account -n istio-system
```
whose output should be
```bash
yes
```
In this command, we can replace verb "get" with "list" or "watch", and the output should always
be "yes". Now let us test with other service accounts.

```bash
kubectl auth can-i get secret/istio-ingress-certs --as system:serviceaccount:istio-system:istio-pilot-service-account -n istio-system
```
whose output should be
```bash
no - Unknown user "system:serviceaccount:istio-system:istio-pilot-service-account"
```
In this command, we can replace service account with istio-mixer-service-account, or
istio-ca-service-account, we can also replace verb "get" with "watch" or "list", and the
output should look similarly.

Accessibility to secret resources except istio-ingress-certs should remain the same for
istio-ca-service-account, istio-ingress-service-account, istio-pilot-service-account and
istio-mixer-service-account.
```bash
kubectl auth can-i get secret/istio-ca-service-account-token-r14xm --as system:serviceaccount:istio-system:istio-ca-service-account -n istio-system
```
whose output should be
```bash
yes
```

## Configuring ingress for gRPC

The ingress controller currently doesn't support `.` characters in the `path`
Expand Down