Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve support for log_statement and log_duration in PostgreSQL module #24607

Merged
merged 10 commits into from
Mar 30, 2021

Conversation

jsoriano
Copy link
Member

@jsoriano jsoriano commented Mar 17, 2021

What does this PR do?

  • Pipeline parses statement durations when statement is not included in the log line.
  • Pipeline parses statements when duration is not included in the log line.
  • Query duration dashboard partially works now when events include durations but no statements.
  • Queries in dashboards filter per event.dataset: postgresql.log for better performance.

Why is it important?

  • Module provides a better experience with a variety of logging settings.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation See Add docs about PostgreSQL monitoring #24588.
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

Use cases

Module works better now when logs include lines generated by statements when log_statement is used and/or log_min_duration_statement is not used, giving a better initial experience in some cases, and allowing different options to configure PostgreSQL and still get relevant data.

@jsoriano jsoriano added enhancement module Filebeat Filebeat needs_backport PR is waiting to be backported to other branches. labels Mar 17, 2021
@jsoriano jsoriano self-assigned this Mar 17, 2021
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Mar 17, 2021
@jsoriano jsoriano added the Team:Integrations Label for the Integrations team label Mar 17, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations (Team:Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Mar 17, 2021
@elasticmachine
Copy link
Collaborator

elasticmachine commented Mar 17, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #24607 updated

  • Start Time: 2021-03-24T18:33:18.060+0000

  • Duration: 59 min 58 sec

  • Commit: 2cef4c7

Test stats 🧪

Test Results
Failed 0
Passed 13175
Skipped 2271
Total 15446

Trends 🧪

Image of Build Times

Image of Tests

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 13175
Skipped 2271
Total 15446

@jsoriano
Copy link
Member Author

/test

CHANGELOG.next.asciidoc Outdated Show resolved Hide resolved
@jsoriano jsoriano merged commit d00dd45 into elastic:master Mar 30, 2021
@jsoriano jsoriano deleted the postgresql-query-dashboard branch March 30, 2021 11:02
jsoriano added a commit to jsoriano/beats that referenced this pull request Mar 30, 2021
…le (elastic#24607)

Provide a better experience with a variety of logging settings:
* Pipeline parses statement durations when statement is not included in the log line.
* Pipeline parses statements when duration is not included in the log line.
* Query duration dashboard partially works now when events include
  durations but no statements.
* Queries in dashboards filter per `event.dataset: postgresql.log` for better performance.

(cherry picked from commit d00dd45)
@jsoriano jsoriano removed the needs_backport PR is waiting to be backported to other branches. label Mar 30, 2021
jsoriano added a commit that referenced this pull request Mar 30, 2021
…le (#24607) (#24841)

Provide a better experience with a variety of logging settings:
* Pipeline parses statement durations when statement is not included in the log line.
* Pipeline parses statements when duration is not included in the log line.
* Query duration dashboard partially works now when events include
  durations but no statements.
* Queries in dashboards filter per `event.dataset: postgresql.log` for better performance.

(cherry picked from commit d00dd45)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants