Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[receiver/k8sobjects] Support running in kubernetes leader election mode. #24662

Conversation

JaredTan95
Copy link
Member

@JaredTan95 JaredTan95 commented Jul 28, 2023

Description: add leader election func to support otel col running in kubernetes leader election mode

BTW, as @dmitryax #17369 (comment), this feature also works for k8scluster, k8sevents receivers, so put this function as an internal method.

Link to tracking Issue: #17369, #32994

Testing:

  1. deploy otel col in 3 replicas, and configuraed k8sobjectsreceiver to watch kubernetes event logs as follows yaml:
apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: default
  name: otel-event-receiver-lr
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: otel-event-receiver-lr
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: otel-event-receiver-lr
subjects:
  - kind: ServiceAccount
    name: otel-event-receiver-lr
    namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: otel-event-receiver-lr
rules:
- apiGroups:
  - coordination.k8s.io
  resources:
  - leases
  verbs:
  - create
  - get
  - update
- apiGroups:
  - ""
  resources:
  - events
  - pods
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - "events.k8s.io"
  resources:
  - events
  verbs:
  - watch
  - list
---
apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: otel-event-receiver-collector
spec:
  replicas: 3
  mode: deployment # This configuration is omittable.
  image: wutang/otelcontribcol:lr-0831 # my onw image build By PR changes.
  serviceAccount: otel-event-receiver-lr
  config: |
    receivers:
      k8sobjects:
        leader_election:
          enabled: true
          lock_name: otel-event-receiver-lr
        auth_type: serviceAccount
        objects:
          - name: events
            mode: watch
            group: events.k8s.io

    exporters:
      logging:
        verbosity: detailed

    service:
      pipelines:
        logs:
          receivers: [k8sobjects]
          exporters: [logging]
  1. check 3 replica pod's logs, there is always one pod in working and watch event lgos:
image

Documentation:

@JaredTan95 JaredTan95 requested a review from a team July 28, 2023 15:26
@JaredTan95 JaredTan95 requested a review from dmitryax as a code owner July 28, 2023 15:26
@github-actions github-actions bot requested a review from hvaghani221 July 28, 2023 15:27
@JaredTan95 JaredTan95 added enhancement New feature or request and removed internal/k8sconfig labels Jul 28, 2023
@JaredTan95 JaredTan95 force-pushed the k8sobjectreceiver_leader_election branch 2 times, most recently from c4cd0b9 to cc9269e Compare July 28, 2023 22:45
@JaredTan95 JaredTan95 closed this Jul 28, 2023
@JaredTan95 JaredTan95 deleted the k8sobjectreceiver_leader_election branch July 28, 2023 22:49
@JaredTan95 JaredTan95 restored the k8sobjectreceiver_leader_election branch July 28, 2023 22:49
@JaredTan95 JaredTan95 reopened this Jul 28, 2023
@JaredTan95 JaredTan95 force-pushed the k8sobjectreceiver_leader_election branch from a888885 to 49846ac Compare July 29, 2023 05:11
@JaredTan95 JaredTan95 force-pushed the k8sobjectreceiver_leader_election branch from 4e234c9 to dd0bbaa Compare July 29, 2023 05:22
Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>
@JaredTan95
Copy link
Member Author

JaredTan95 commented Aug 2, 2023

@jpkrohling @dmitryax hi, do you have time to help review this PR?

.chloggen/support-kubernetes-leader-election.yaml Outdated Show resolved Hide resolved
receiver/k8sobjectsreceiver/config_test.go Outdated Show resolved Hide resolved
receiver/k8sobjectsreceiver/receiver.go Show resolved Hide resolved
receiver/k8sobjectsreceiver/factory.go Outdated Show resolved Hide resolved
@github-actions
Copy link
Contributor

This PR was marked stale due to lack of activity. It will be closed in 14 days.

Copy link
Contributor

This PR was marked stale due to lack of activity. It will be closed in 14 days.

@github-actions github-actions bot added the Stale label Nov 14, 2023
@jpkrohling
Copy link
Member

What's the state of this PR?

@JaredTan95
Copy link
Member Author

What's the state of this PR?

finding a way to add Unit-test...

@github-actions github-actions bot removed the Stale label Nov 28, 2023
@JaredTan95 JaredTan95 marked this pull request as draft December 10, 2023 05:25
Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>
Copy link
Contributor

This PR was marked stale due to lack of activity. It will be closed in 14 days.

Copy link
Contributor

github-actions bot commented Jan 9, 2024

This PR was marked stale due to lack of activity. It will be closed in 14 days.

Copy link
Contributor

This PR was marked stale due to lack of activity. It will be closed in 14 days.

@github-actions github-actions bot added the Stale label Jan 24, 2024
Copy link
Contributor

github-actions bot commented Feb 8, 2024

Closed as inactive. Feel free to reopen if this PR is still being worked on.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants