Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Routes: regex doesn't work #4763

Closed
vasily-yanov-draivn opened this issue Jul 24, 2024 · 1 comment · Fixed by #4798
Closed

[BUG] Routes: regex doesn't work #4763

vasily-yanov-draivn opened this issue Jul 24, 2024 · 1 comment · Fixed by #4798
Assignees
Labels
bug Something isn't working
Milestone

Comments

@vasily-yanov-draivn
Copy link

Describe the bug

create pipeline with a route:

            - dev-nginx-logs: '/kubernetes/pod_name =~ ".*nginx.*" and /k8scluster == "dev-cluster"'
            - dev-app-logs: '/kubernetes/namespace_name == "my-app-dev"'

but when DP starts I see:

2024-07-24T14:52:59,685 [logs-pipeline-processor-worker-1-thread-3] ERROR org.opensearch.dataprepper.pipeline.router.RouteEventEvaluator - Failed to evaluate route. This route will not be applied to any events.
org.opensearch.dataprepper.expression.ExpressionEvaluationException: Unable to evaluate statement "".*nginx.*" and /k8scluster == "dev-cluster""
        at org.opensearch.dataprepper.expression.GenericExpressionEvaluator.evaluate(GenericExpressionEvaluator.java:44) ~[data-prepper-expression-2.8.0.jar:?]
        at org.opensearch.dataprepper.expression.ExpressionEvaluator.evaluateConditional(ExpressionEvaluator.java:30) ~[data-prepper-api-2.8.0.jar:?]
        at org.opensearch.dataprepper.pipeline.router.RouteEventEvaluator.findMatchedRoutes(RouteEventEvaluator.java:64) [data-prepper-core-2.8.0.jar:?]
        at org.opensearch.dataprepper.pipeline.router.RouteEventEvaluator.evaluateEventRoutes(RouteEventEvaluator.java:45) [data-prepper-core-2.8.0.jar:?]
        at org.opensearch.dataprepper.pipeline.router.Router.route(Router.java:44) [data-prepper-core-2.8.0.jar:?]
        at org.opensearch.dataprepper.pipeline.Pipeline.publishToSinks(Pipeline.java:346) [data-prepper-core-2.8.0.jar:?]
        at org.opensearch.dataprepper.pipeline.ProcessWorker.postToSink(ProcessWorker.java:168) [data-prepper-core-2.8.0.jar:?]
        at org.opensearch.dataprepper.pipeline.ProcessWorker.doRun(ProcessWorker.java:150) [data-prepper-core-2.8.0.jar:?]
        at org.opensearch.dataprepper.pipeline.ProcessWorker.run(ProcessWorker.java:61) [data-prepper-core-2.8.0.jar:?]
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
        at java.base/java.lang.Thread.run(Thread.java:840) [?:?]
Caused by: org.opensearch.dataprepper.expression.ExpressionEvaluationException: Unable to evaluate the part of input statement: /kubernetes/pod_name =~ ".*nginx.*"
        at org.opensearch.dataprepper.expression.ParseTreeEvaluator.evaluate(ParseTreeEvaluator.java:41) ~[data-prepper-expression-2.8.0.jar:?]
        at org.opensearch.dataprepper.expression.ParseTreeEvaluator.evaluate(ParseTreeEvaluator.java:17) ~[data-prepper-expression-2.8.0.jar:?]
        at org.opensearch.dataprepper.expression.GenericExpressionEvaluator.evaluate(GenericExpressionEvaluator.java:41) ~[data-prepper-expression-2.8.0.jar:?]
        ... 13 more
.......
Caused by: java.lang.IllegalArgumentException: '=~' requires left operand to be String.

To Reproduce

  1. Create pipeline with route provided above
  2. Check

Expected behavior
route events to indexes according route rules

Environment (please complete the following information):

  • AWS EKS
  • Version 2.8.0
@vasily-yanov-draivn vasily-yanov-draivn added bug Something isn't working untriaged labels Jul 24, 2024
@dlvenable
Copy link
Member

From my investigation of this, the problem is that the regex operation fails when the key is not found in the event. Basically, when /kubernetes/pod_name is null, this throws a failure. I will create a PR for this.

@dlvenable dlvenable self-assigned this Aug 1, 2024
@dlvenable dlvenable added this to the v2.9 milestone Aug 1, 2024
dlvenable added a commit to dlvenable/data-prepper that referenced this issue Aug 1, 2024
…n is null, always return false rather than throwing an exception. Resolves opensearch-project#4763.

Signed-off-by: David Venable <[email protected]>
dlvenable added a commit that referenced this issue Aug 12, 2024
…n is null, always return false rather than throwing an exception. Resolves #4763. (#4798)

Signed-off-by: David Venable <[email protected]>
opensearch-trigger-bot bot pushed a commit that referenced this issue Aug 12, 2024
…n is null, always return false rather than throwing an exception. Resolves #4763. (#4798)

Signed-off-by: David Venable <[email protected]>
(cherry picked from commit 176d828)
kkondaka pushed a commit to kkondaka/kk-data-prepper-f2 that referenced this issue Aug 12, 2024
…n is null, always return false rather than throwing an exception. Resolves opensearch-project#4763. (opensearch-project#4798)

Signed-off-by: David Venable <[email protected]>
Signed-off-by: Krishna Kondaka <[email protected]>
kkondaka pushed a commit to kkondaka/kk-data-prepper-f2 that referenced this issue Aug 12, 2024
…n is null, always return false rather than throwing an exception. Resolves opensearch-project#4763. (opensearch-project#4798)

Signed-off-by: David Venable <[email protected]>
Signed-off-by: Krishna Kondaka <[email protected]>
kkondaka pushed a commit to kkondaka/kk-data-prepper-f2 that referenced this issue Aug 14, 2024
…n is null, always return false rather than throwing an exception. Resolves opensearch-project#4763. (opensearch-project#4798)

Signed-off-by: David Venable <[email protected]>
Signed-off-by: Krishna Kondaka <[email protected]>
kkondaka pushed a commit to kkondaka/kk-data-prepper-f2 that referenced this issue Aug 14, 2024
…n is null, always return false rather than throwing an exception. Resolves opensearch-project#4763. (opensearch-project#4798)

Signed-off-by: David Venable <[email protected]>
Signed-off-by: Krishna Kondaka <[email protected]>
kkondaka pushed a commit to kkondaka/kk-data-prepper-f2 that referenced this issue Aug 14, 2024
…n is null, always return false rather than throwing an exception. Resolves opensearch-project#4763. (opensearch-project#4798)

Signed-off-by: David Venable <[email protected]>
Signed-off-by: Krishna Kondaka <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants