Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

detectExceptions is not forbidden in multi-worker setups #1490

Open
florianstoeber opened this issue Sep 27, 2023 · 1 comment
Open

detectExceptions is not forbidden in multi-worker setups #1490

florianstoeber opened this issue Sep 27, 2023 · 1 comment
Milestone

Comments

@florianstoeber
Copy link
Contributor

florianstoeber commented Sep 27, 2023

Describe the bug:
If I have a logging-instance running with multiple fluentd workers it is not forbidden to use the detectExceptions filter, while this will break the fluentd processes.

Expected behaviour:
As this is a known limitation, it would be nice to mark the Flow as "not valid".

Steps to reproduce the bug:
Create a logging-instance with multiple workers and create a Flow with detectExceptions filter.

Environment details:

  • Kubernetes version (e.g. v1.15.2): 1.25
  • Cloud-provider/provisioner (e.g. AKS, GKE, EKS, PKE etc): AKS
  • logging-operator version (e.g. 2.1.1): 4.2.2
  • Install method (e.g. helm or static manifests): Helm
  • Logs from the misbehaving component (and any other relevant logs):
  • Resource definition (possibly in YAML format) that caused the issue, without sensitive data:
apiVersion: logging.banzaicloud.io/v1beta1
kind: Flow
metadata:
  name: flow
spec:
  filters:
  - record_modifier:
      records:
      - fluentd_worker: ${ENV['HOSTNAME']}
  - record_transformer:
      remove_keys: $.kubernetes.docker_id, $.kubernetes.annotations, $.kubernetes.container_hash,
        $.kubernetes.pod_id
  - parser:
      key_name: message
      parse:
        type: json
      remove_key_name_field: true
      reserve_data: true
  - detectExceptions:
      languages:
      - java
      - python
      multiline_flush_interval: "0.1"
  localOutputRefs:
  - output
  match:
  - select: {}

/kind bug

@florianstoeber florianstoeber added the bug Something isn't working label Sep 27, 2023
@pepov pepov added enhancement New feature or request pinned and removed bug Something isn't working labels Oct 5, 2023
@pepov
Copy link
Member

pepov commented Oct 5, 2023

The request is valid, although this is not a bug, but an enhancement request. It adds a validation on top of fluentd that doesn't tell us about the problem exactly.

@pepov pepov added the fluentd label Oct 5, 2023
@pepov pepov added this to the 4.x milestone Oct 5, 2023
@csatib02 csatib02 modified the milestones: 5.x, Fluentd Nov 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants