Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix topk and bottomk operations with int <= 0 #6937

Merged
merged 5 commits into from
Aug 19, 2022
Merged

Conversation

ssncferreira
Copy link
Contributor

@ssncferreira ssncferreira commented Aug 19, 2022

What this PR does / why we need it:

How the problem was identified
Range mapper query fails for topk/bottomk expressions with parameter 0.
When parsing an expression the current flow is:

  1. ParseSampleExpr from query string: https://github.com/grafana/loki/blob/main/pkg/logql/rangemapper.go#L80
	origExpr, err := syntax.ParseSampleExpr(query)

this operation succeeds with no error
2) ParseSampleExpr from expr.String(): https://github.com/grafana/loki/blob/main/pkg/logql/rangemapper.go#L451

	e, err := syntax.ParseSampleExpr(expr.String())

expr.String() does not consider the topk parameter 0 because of this condition: https://github.com/grafana/loki/blob/main/pkg/logql/syntax/ast.go#L927-L931

	if e.Params != 0 {
		params = []string{fmt.Sprintf("%d", e.Params), e.Left.String()}
	} else {
		params = []string{e.Left.String()}
	}

incorrectly converting the expression to string topk(sum by(geoip_country_code)(count_over_time({app="foo"} | json | ( geoip_country_code!="" , __error__="" )[15m]))) which then panic at https://github.com/grafana/loki/blob/main/pkg/logql/syntax/ast.go#L869-L871 because param is nil

		if params == nil {
			panic(logqlmodel.NewParseError(fmt.Sprintf("parameter required for operation %s", operation), 0, 0))
		}

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:

The changes in this PR:

  1. panic if the param is smaller or equal to 0, returning an error immediately on step 1.
  2. fix expr.String() to consider the first parameter of topk/bottomk even if 0

Checklist

  • Documentation added
  • Tests updated
  • Is this an important fix or new feature? Add an entry in the CHANGELOG.md.
  • Changes that require user attention or interaction to upgrade are documented in docs/sources/upgrading/_index.md

@grafanabot
Copy link
Collaborator

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

@grafanabot
Copy link
Collaborator

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

@pull-request-size pull-request-size bot added size/M and removed size/S labels Aug 19, 2022
@ssncferreira ssncferreira marked this pull request as ready for review August 19, 2022 10:13
@ssncferreira ssncferreira requested a review from a team as a code owner August 19, 2022 10:13
@grafanabot
Copy link
Collaborator

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

Copy link
Contributor

@MichelHollands MichelHollands left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice, thanks for fixing this.

@grafanabot
Copy link
Collaborator

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

Copy link
Member

@owen-d owen-d left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nice explanation & fix

@owen-d owen-d merged commit 3f3f4ed into main Aug 19, 2022
@owen-d owen-d deleted the fix_topk_invalid_parameter branch August 19, 2022 12:47
lxwzy pushed a commit to lxwzy/loki that referenced this pull request Nov 7, 2022
* Fix topk and bottomk operations with int <= 0

* Fix VectorAggregationExpr String method

* Update rangemapper_test expression

* Add tests

* Update CHANGELOG.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants