Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[chore] add documentation for setting up RBAC for the k8sobserver extension #35597

Merged
merged 2 commits into from
Oct 11, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
167 changes: 166 additions & 1 deletion extension/observer/k8sobserver/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,4 +75,169 @@ All fields are optional.
| observe_ingresses | bool | `false` | Whether to report observer k8s.ingress endpoints.|

More complete configuration examples on how to use this observer along with the `receiver_creator`,
can be found at the [Receiver Creator](../../../receiver/receivercreator/README.md)'s documentation.
can be found at the [Receiver Creator](../../../receiver/receivercreator/README.md)'s documentation.

### Setting up RBAC permissions

When using the `serviceAccount` `auth_type`, the service account of the pod running the agent needs to have the required permissions to
read the K8s resources it should observe (i.e. pods, nodes, services and ingresses).
Therefore, the service account running the pod needs to have the required `ClusterRole` which grants it the permission to
read those resources from the Kubernetes API. Below is an example of how to set this up:

1. Create a `ServiceAccount` that the collector should use.

```bash
<<EOF | kubectl apply -f -
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
app: otelcontribcol
name: otelcontribcol
EOF
```

2. Create a `ClusterRole`/`ClusterRoleBinding` that grants permission to read pods, nodes, services and ingresses.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose using just a Role instead of ClusterRole is also an option if the user only want to observe e.g. pods in a specific namespace?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the late reply - right now, the k8sobserver works only with ClusterRoles, but I'm working on an issue (#9401) to also support the use of namespaced resources and thus also be usable with Roles instead of ClusterRoles

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Edit: I just realized that the issue i linked in the comment above is for the k8sclusterreceiver, and not the k8sobserver - However, looking at the code of the k8sobserver, it seems like the informer being created here also is currently always attempting to observe all namespaces. so it might make sense to also do something similar as in #9401 for this component.


Note: If you do not plan to observe all of these resources (e.g. if you are only interested in services) it is recommended to remove
the resources you do not intend to observe from the configuration below:

```bash
<<EOF | kubectl apply -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: otelcontribcol
labels:
app: otelcontribcol
rules:
- apiGroups:
- ""
resources:
- nodes
- services
- pods
verbs:
- get
- list
- watch
- apiGroups:
- "networking.k8s.io"
resources:
- ingresses
verbs:
- get
- watch
- list
EOF
```

```bash
<<EOF | kubectl apply -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: otelcontribcol
labels:
app: otelcontribcol
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: otelcontribcol
subjects:
- kind: ServiceAccount
name: otelcontribcol
namespace: default
EOF
```
3. Create a ConfigMap containing the configuration for the collector

```bash
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
name: otelcontribcol
labels:
app: otelcontribcol
data:
config.yaml: |
extensions:
k8s_observer:
auth_type: serviceAccount
node: ${env:K8S_NODE_NAME}
observe_pods: true
observe_nodes: true
observe_services: true
observe_ingresses: true

receivers:
receiver_creator:
watch_observers: [k8s_observer]
receivers:
redis:
rule: type == "port" && pod.name matches "redis"
config:
password: '`pod.labels["SECRET"]`'
kubeletstats:
rule: type == "k8s.node"
config:
auth_type: serviceAccount
collection_interval: 10s
endpoint: "`endpoint`:`kubelet_endpoint_port`"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using kubelet_endpoint_port to monitor the discovered Redis Pod looks weird? Should be just port instead?

Suggested change
endpoint: "`endpoint`:`kubelet_endpoint_port`"
endpoint: "`endpoint`:`port`"

ref: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/receivercreator/README.md#port

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just looked into that again - i think this still needs to be kubelet_endpoint_port, as this is not related to the redis receiver example, but for a kubeletstats receiver, which is created based on a detected k8s.node (this example is essentially a copy paste from the config example further above in the readme).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I just realized that there are 2 different config blocks, I saw it as only one 🤦🏽‍♂️

extra_metadata_labels:
- container.id
metric_groups:
- container
- pod
- node

exporters:
otlp:
endpoint: <OTLP_ENDPOINT>

service:
pipelines:
metrics:
receivers: [receiver_creator]
exporters: [otlp]
EOF
```

4. Create the collector deployment, referring to the service account created earlier

```bash
<<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
name: otelcontribcol
labels:
app: otelcontribcol
spec:
replicas: 1
selector:
matchLabels:
app: otelcontribcol
template:
metadata:
labels:
app: otelcontribcol
spec:
serviceAccountName: otelcontribcol
containers:
- name: otelcontribcol
# This image is created by running `make docker-otelcontribcol`.
# If you are not building the collector locally, specify a published image: `otel/opentelemetry-collector-contrib`
image: otelcontribcol:latest
args: ["--config", "/etc/config/config.yaml"]
volumeMounts:
- name: config
mountPath: /etc/config
imagePullPolicy: IfNotPresent
volumes:
- name: config
configMap:
name: otelcontribcol
EOF
```