Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Filebeat] Fix date parsing in GSuite/Google Workspace modules #24696

Merged
merged 2 commits into from
Mar 23, 2021

Conversation

andrewkroh
Copy link
Member

@andrewkroh andrewkroh commented Mar 22, 2021

What does this PR do?

The format of Date#toUTCString was incompatible with the format accepted by Elasticsearch by default.
By writing a Date object from the JS pipeline this becomes a time.Time in the event that is formatted by
common.Time when going out as JSON.

Fixes #24694
Fixes #24692

Why is it important?

Fixes data ingesting exceptions and prevents event loss.

Checklist

  • My code follows the style guidelines of this project
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

@andrewkroh andrewkroh added bug Filebeat Filebeat needs_backport PR is waiting to be backported to other branches. Team:Security-External Integrations labels Mar 22, 2021
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Mar 22, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Mar 22, 2021
@elasticmachine
Copy link
Collaborator

elasticmachine commented Mar 22, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: andrewkroh commented: run tests

  • Start Time: 2021-03-23T19:16:16.780+0000

  • Duration: 54 min 39 sec

  • Commit: a3f22c0

Test stats 🧪

Test Results
Failed 0
Passed 6847
Skipped 1169
Total 8016

Trends 🧪

Image of Build Times

Image of Tests

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 6847
Skipped 1169
Total 8016

@legoguy1000
Copy link
Contributor

@andrewkroh Looks like this same fix is needed https://github.com/elastic/beats/blob/d427453a9f311e00f91354da03fd3bbb99d82e50/x-pack/filebeat/module/google_workspace/login/config/pipeline.js#L67 to resolve #24692

@andrewkroh andrewkroh changed the title [Filebeat] Fix date parsing in GSuite/login fileset [Filebeat] Fix date parsing in GSuite/Google Workspace modules Mar 23, 2021
The format of Date#toUTCString was incompatible with the format accepted by Elasticsearch by default.
By writing a Date object from the JS pipeline this becomes a time.Time in the event that is formatted by
common.Time when going out as JSON.

Fixes elastic#24694
@andrewkroh andrewkroh force-pushed the bugfix/fb/gsuite-login-date branch from 954e99f to a3f22c0 Compare March 23, 2021 14:03
@andrewkroh
Copy link
Member Author

run tests

@andrewkroh andrewkroh merged commit a4a3ff0 into elastic:master Mar 23, 2021
@andrewkroh andrewkroh removed the needs_backport PR is waiting to be backported to other branches. label Mar 24, 2021
andrewkroh added a commit to andrewkroh/beats that referenced this pull request Mar 24, 2021
…ic#24696)

* Fix date parsing in GSuite/login fileset

The format of Date#toUTCString was incompatible with the format accepted by Elasticsearch by default.
By writing a Date object from the JS pipeline this becomes a time.Time in the event that is formatted by
common.Time when going out as JSON.

Fixes elastic#24694

* Apply fix to google_workspace

Fixes elastic#24692

(cherry picked from commit a4a3ff0)
andrewkroh added a commit to andrewkroh/beats that referenced this pull request Mar 24, 2021
…ic#24696)

* Fix date parsing in GSuite/login fileset

The format of Date#toUTCString was incompatible with the format accepted by Elasticsearch by default.
By writing a Date object from the JS pipeline this becomes a time.Time in the event that is formatted by
common.Time when going out as JSON.

Fixes elastic#24694

* Apply fix to google_workspace

Fixes elastic#24692

(cherry picked from commit a4a3ff0)
andrewkroh added a commit that referenced this pull request Mar 24, 2021
… (#24735)

* Fix date parsing in GSuite/login fileset

The format of Date#toUTCString was incompatible with the format accepted by Elasticsearch by default.
By writing a Date object from the JS pipeline this becomes a time.Time in the event that is formatted by
common.Time when going out as JSON.

Fixes #24694

* Apply fix to google_workspace

Fixes #24692

(cherry picked from commit a4a3ff0)
andrewkroh added a commit that referenced this pull request Mar 24, 2021
… (#24736)

* Fix date parsing in GSuite/login fileset

The format of Date#toUTCString was incompatible with the format accepted by Elasticsearch by default.
By writing a Date object from the JS pipeline this becomes a time.Time in the event that is formatted by
common.Time when going out as JSON.

Fixes #24694

* Apply fix to google_workspace

Fixes #24692

(cherry picked from commit a4a3ff0)
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
…ic#24696) (elastic#24736)

* Fix date parsing in GSuite/login fileset

The format of Date#toUTCString was incompatible with the format accepted by Elasticsearch by default.
By writing a Date object from the JS pipeline this becomes a time.Time in the event that is formatted by
common.Time when going out as JSON.

Fixes elastic#24694

* Apply fix to google_workspace

Fixes elastic#24692

(cherry picked from commit 2a63518)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Filebeat] Parsing issue in gsuite module [Filebeat] Google workspace parsing issue
4 participants