Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: trace V4 QB #6407

Merged
merged 10 commits into from
Nov 13, 2024
Merged

feat: trace V4 QB #6407

merged 10 commits into from
Nov 13, 2024

Conversation

nityanandagohain
Copy link
Member

@nityanandagohain nityanandagohain commented Nov 8, 2024

  • Uses the resource QB for resource filters.
  • Common format for mat columns apart from the static ones i.e <name`
    • eg:- attribute_string_message
  • The usecase of StaticFieldsTraces will be more visible in the future PR's for enrichment.

Part of #5713


Important

Introduces v4 query builder for traces with resource filters, common column format, and enhanced query construction and testing.

  • Behavior:
    • Introduces v4/query_builder.go for handling trace queries with resource filters and a common format for materialized columns.
    • Uses resource package for building resource subqueries in v4/query_builder.go.
    • Adds support for static fields in constants.go for trace queries.
  • Functions:
    • Adds getClickHouseTracesColumnType, getClickHouseTracesColumnDataType, and getColumnName in v4/query_builder.go for column handling.
    • Implements buildTracesFilterQuery and handleEmptyValuesInGroupBy in v4/query_builder.go for query construction.
    • Introduces orderBy and orderByAttributeKeyTags in v4/query_builder.go for ordering logic.
  • Tests:
    • Adds query_builder_test.go for v4 with tests for new functions like getClickHouseTracesColumnType, getColumnName, and buildTracesQuery.
  • Misc:
    • Exposes AggregateOperatorToPercentile and AggregateOperatorToSQLFunc in v3/query_builder.go for reuse in v4.
    • Updates constants.go with new constants for v3 and v4 trace handling.

This description was created by Ellipsis for d399a9a. It will automatically update as commits are pushed.

@github-actions github-actions bot added docs required enhancement New feature or request labels Nov 8, 2024
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Reviewed everything up to d0595bd in 1 minute and 2 seconds

More details
  • Looked at 1350 lines of code in 4 files
  • Skipped 0 files when reviewing.
  • Skipped posting 3 drafted comments based on config settings.
1. pkg/query-service/constants/constants.go:450
  • Draft comment:
    The StaticFieldsTraces map is defined twice, once here and once in query_builder.go. Consider defining it in one place to avoid inconsistencies.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The PR introduces a new map StaticFieldsTraces in constants.go which is used in query_builder.go to check if a key is a static field. This map is used to determine if a key should be returned as is in the getColumnName function. This is a good approach to handle static fields, but it should be noted that the map is defined twice, once in constants.go and once in query_builder.go. This could lead to inconsistencies if the maps are not kept in sync. It would be better to define the map in one place and reuse it.
2. pkg/query-service/app/traces/v4/query_builder.go:1
  • Draft comment:
    Avoid using the component/index.tsx file structure approach, as it makes it difficult to debug and find components using global search tools like VS Code.
  • Reason this comment was not posted:
    Comment was on unchanged code.
3. pkg/query-service/app/traces/v3/query_builder.go:25
  • Draft comment:
    This map is already defined in pkg/query-service/app/logs/v3/query_builder.go. Consider using the existing map to avoid duplication.

  • constant AggregateOperatorToSQLFunc (query_builder.go)

  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable:
    The comment is about a potential code quality issue (duplication) which is generally useful feedback. However, the map 'AggregateOperatorToSQLFunc' was not newly introduced in this diff; it was only made public. The comment does not directly relate to a change made in this diff, as the map already existed in the file before the change.
    The comment might still be useful for future refactoring, but it does not address a change made in this diff. The rules specify that comments should be about changes made in the diff.
    While the comment is about a potential improvement, it does not pertain to a change made in this diff, so it should be removed according to the rules.
    Remove the comment because it does not pertain to a change made in this diff, even though it highlights a potential code quality improvement.

Workflow ID: wflow_6wbzX5RuVCox9yT2


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@srikanthccv
Copy link
Member

@nityanandagohain this is big PR. Please do it parts.

@nityanandagohain
Copy link
Member Author

I have kept just the two major files. Do suggest to break pr's down function wise ?

@srikanthccv
Copy link
Member

Screenshot 2024-11-12 at 1 18 41 PM

These can be independently merged?

@nityanandagohain
Copy link
Member Author

  1. So the first one is just a single line i.e to add the filters.
resourceSubQuery, err := resource.BuildResourceSubQuery("signoz_traces", "distributed_traces_v3_resource", bucketStart, bucketEnd, mq.Filters, mq.GroupBy, mq.AggregateAttribute, false)
  1. This is required i.e the change in columnName.
  2. For supporting 2, I have added 3

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on 4637dc1 in 44 seconds

More details
  • Looked at 39 lines of code in 2 files
  • Skipped 0 files when reviewing.
  • Skipped posting 2 drafted comments based on config settings.
1. pkg/query-service/app/traces/v4/query_builder.go:259
  • Draft comment:
    Removing the id column from the select statement for PanelTypeList might affect functionality if id was required. Verify this change to ensure it doesn't break existing functionality.
  • Reason this comment was not posted:
    Comment did not seem useful.
2. pkg/query-service/app/traces/v4/query_builder.go:56
  • Draft comment:
    Avoid using inline styles in React components. Instead, use external stylesheets, CSS classes, or styled components. This comment also applies to other instances in the file.
  • Reason this comment was not posted:
    Comment was on unchanged code.

Workflow ID: wflow_AGtK3eSCmU03c4D3


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

Copy link
Member

@srikanthccv srikanthccv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add more tests. I would suggest taking couple of production dashboards and more sanity testing to test the valid query preparation and correctness.

pkg/query-service/app/traces/v4/query_builder.go Outdated Show resolved Hide resolved
pkg/query-service/constants/constants.go Show resolved Hide resolved
pkg/query-service/constants/constants.go Show resolved Hide resolved
pkg/query-service/app/traces/v4/query_builder.go Outdated Show resolved Hide resolved
pkg/query-service/app/traces/v4/query_builder_test.go Outdated Show resolved Hide resolved
pkg/query-service/app/traces/v4/query_builder_test.go Outdated Show resolved Hide resolved
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on e8705b3 in 34 seconds

More details
  • Looked at 140 lines of code in 2 files
  • Skipped 0 files when reviewing.
  • Skipped posting 4 drafted comments based on config settings.
1. pkg/query-service/app/traces/v4/query_builder.go:35
  • Draft comment:
    The function getClickHouseTracesColumnType is correctly implemented. Ensure that all possible AttributeKeyType values are covered in the test cases.
  • Reason this comment was not posted:
    Confidence changes required: 10%
    The function getClickHouseTracesColumnType is straightforward, but the test case names could be more descriptive. However, the implementation is correct.
2. pkg/query-service/app/traces/v4/query_builder.go:32
  • Draft comment:
    The change from NOT has(%s%s, '%s') to NOT mapContains(%s, '%s') for FilterOperatorNotExists is correct and consistent with the logic for FilterOperatorExists.
  • Reason this comment was not posted:
    Confidence changes required: 10%
    The change in line 32 of tracesOperatorMappingV3 is correct, as it aligns with the logic for FilterOperatorExists.
3. pkg/query-service/app/traces/v4/query_builder.go:107
  • Draft comment:
    The use of utils.QuoteEscapedStringForContains in line 107 is appropriate for handling special characters in contains operations.
  • Reason this comment was not posted:
    Confidence changes required: 10%
    The change in line 107 is correct as it uses a specialized function for escaping strings in contains operations.
4. pkg/query-service/app/traces/v4/query_builder.go:103
  • Draft comment:
    Avoid using inline styles in React components. Instead, use external stylesheets, CSS classes, or styled components.
  • Reason this comment was not posted:
    Comment was on unchanged code.

Workflow ID: wflow_Oy9qkzcCkJs3Kfst


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on fd9dd52 in 36 seconds

More details
  • Looked at 27 lines of code in 2 files
  • Skipped 0 files when reviewing.
  • Skipped posting 2 drafted comments based on config settings.
1. pkg/query-service/app/logs/v4/query_builder.go:438
  • Draft comment:
    The removal of the step calculation for the table panel type might affect existing functionality. Ensure that this change aligns with the intended behavior and does not introduce any regressions.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The removal of the step calculation for the table panel type in both logs and traces query builders seems intentional. However, it's important to ensure that this change does not affect any existing functionality or assumptions in the codebase.
2. pkg/query-service/app/traces/v4/query_builder.go:292
  • Draft comment:
    The removal of the step calculation for the table panel type might affect existing functionality. Ensure that this change aligns with the intended behavior and does not introduce any regressions.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The removal of the step calculation for the table panel type in both logs and traces query builders seems intentional. However, it's important to ensure that this change does not affect any existing functionality or assumptions in the codebase.

Workflow ID: wflow_vCJf6JkxLkYW7iYE


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on 789aa0d in 40 seconds

More details
  • Looked at 195 lines of code in 3 files
  • Skipped 0 files when reviewing.
  • Skipped posting 2 drafted comments based on config settings.
1. pkg/query-service/app/traces/v4/query_builder.go:52
  • Draft comment:
    The check for static fields is redundant as it is performed twice. Consider moving the static fields check after the column check to avoid redundancy.
  • Reason this comment was not posted:
    Comment was on unchanged code.
2. pkg/query-service/app/traces/v4/query_builder.go:50
  • Draft comment:
    Avoid using inline styles in React components. Use external stylesheets, CSS classes, or styled components instead.
  • Reason this comment was not posted:
    Comment was on unchanged code.

Workflow ID: wflow_VsK5G3KX7eXY7GQ7


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on 0dc76d1 in 34 seconds

More details
  • Looked at 35 lines of code in 1 files
  • Skipped 0 files when reviewing.
  • Skipped posting 2 drafted comments based on config settings.
1. pkg/query-service/app/traces/v4/query_builder_test.go:503
  • Draft comment:
    Test case names should be unique and descriptive to easily identify which test case failed. Consider renaming duplicate test cases like "Test buildTracesQuery" to something more specific.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The test case names are not descriptive enough, especially when there are multiple tests with the same name. This can lead to confusion when tests fail, as it will be difficult to identify which specific test case failed.
2. pkg/query-service/app/traces/v4/query_builder_test.go:499
  • Draft comment:
    Avoid using the component/index.tsx file structure approach, as it makes it difficult to debug and find components using global search tools like VS Code.
  • Reason this comment was not posted:
    Comment was on unchanged code.

Workflow ID: wflow_S7kxZS3jCVC5AN1h


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on d399a9a in 1 minute and 44 seconds

More details
  • Looked at 7733 lines of code in 74 files
  • Skipped 1 files when reviewing.
  • Skipped posting 2 drafted comments based on config settings.
1. pkg/query-service/rules/templates.go:265
  • Draft comment:
    The regex for handling {{$variable}} syntax should not skip special cases for {{$threshold}} and {{$value}} as they are already handled in the template functions.
  • Reason this comment was not posted:
    Comment looked like it was already resolved.
2. pkg/query-service/rules/templates.go:265
  • Draft comment:
    The preprocessTemplate function is incorrectly handling the {{$variable}} syntax by not skipping the special cases for {{$threshold}} and {{$value}}. This could lead to incorrect template processing. Ensure these special cases are skipped.
  • Reason this comment was not posted:
    Marked as duplicate.

Workflow ID: wflow_8Q56L5Cmifq9Elw8


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@nityanandagohain nityanandagohain merged commit 2faa0c6 into develop Nov 13, 2024
18 of 19 checks passed
@nityanandagohain nityanandagohain deleted the feat/trace-v4-qb branch November 13, 2024 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs not required enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants