Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[crowdstrike] Create FDR and SIEM Connector policy templates #2806

Closed

Conversation

andrewkroh
Copy link
Member

@andrewkroh andrewkroh commented Mar 9, 2022

DRAFT: Integration upgrades are not working.

What does this PR do?

This creates two new integrations in Fleet. One specifically for Falcon Data Replicator
and one for Falcon SIEM Connector. These already existed in the "CrowdStrike"
integration, but now they will also be listed on their own from the Fleet integration
pages to make them more discoverable to users.

There will still be a "CrowdStrike Logs" integration that contains inputs for all both
FDR and the SIEM connector. This is similar to how the AWS integration contains
inputs from all of the individual AWS services.

This also removes the logfile input from the FDR data stream since it is not usable
given the FDR tool writes gzip compressed files to disk. Only the AWS S3 input will
be available. This addresses #2194.

Screen Shot 2022-03-09 at 5 08 36 PM

Screen Shot 2022-03-09 at 5 08 32 PM

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Related issues

Screenshots

Falcon SIEM Connector

Screen Shot 2022-03-09 at 5 09 13 PM

Screen Shot 2022-03-09 at 5 09 06 PM

FDR

Screen Shot 2022-03-09 at 5 08 56 PM

Screen Shot 2022-03-09 at 5 08 43 PM

Upgrade Error

Error: Root value is not flatten-able, received undefined

Screen Shot 2022-03-09 at 4 52 34 PM

{
  "type": "log",
  "@timestamp": "2022-03-09T22:04:23+00:00",
  "tags": [
    "info",
    "plugins",
    "fleet"
  ],
  "pid": 7,
  "package_policy_upgrade": {
    "package_name": "crowdstrike",
    "current_version": "1.2.3",
    "new_version": "1.3.2",
    "status": "failure",
    "error": [
      {
        "key": "inputs.crowdstrike_fdr-aws-s3.streams.crowdstrike.fdr.vars.queue_url",
        "message": [
          "Queue URL is required"
        ]
      }
    ],
    "dryRun": true
  },
  "message": "Package policy upgrade dry run resulted in errors {\"package_policy_upgrade\":{\"package_name\":\"crowdstrike\",\"current_version\":\"1.2.3\",\"new_version\":\"1.3.2\",\"status\":\"failure\",\"error\":[{\"key\":\"inputs.crowdstrike_fdr-aws-s3.streams.crowdstrike.fdr.vars.queue_url\",\"message\":[\"Queue URL is required\"]}],\"dryRun\":true}}"
}

@elasticmachine
Copy link

elasticmachine commented Mar 9, 2022

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-07-27T18:46:52.491+0000

  • Duration: 14 min 13 sec

Steps errors 2

Expand to view the steps failures

Test integration: crowdstrike
  • Took 1 min 30 sec . View more details here
  • Description: eval "$(../../build/elastic-package stack shellinit)" ../../build/elastic-package test -v --report-format xUnit --report-output file --test-coverage
Google Storage Download
  • Took 0 min 0 sec . View more details here

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

This creates two new integrations in Fleet. One specifically for Falcon Data Replicator
and one for Falcon SIEM Connector. These already existed in the "CrowdStrike"
integration, but now they will also be listed on their own from the Fleet integration
pages to make them more discoverable to users.

There will still be a "CrowdStrike Logs" integration that contains inputs for all both
FDR and the SIEM connector. This is similar to how the AWS integration contains
inputs from all of the individual AWS services.

This also removes the logfile input from the FDR data stream since it is not usable
given the FDR tool writes gzip compressed files to disk. Only the AWS S3 input will
be available. This addresses elastic#2194.
Upgrades are broken. The error is

{
  "type": "log",
  "@timestamp": "2022-03-09T22:04:23+00:00",
  "tags": [
    "info",
    "plugins",
    "fleet"
  ],
  "pid": 7,
  "package_policy_upgrade": {
    "package_name": "crowdstrike",
    "current_version": "1.2.3",
    "new_version": "1.3.2",
    "status": "failure",
    "error": [
      {
        "key": "inputs.crowdstrike_fdr-aws-s3.streams.crowdstrike.fdr.vars.queue_url",
        "message": [
          "Queue URL is required"
        ]
      }
    ],
    "dryRun": true
  },
  "message": "Package policy upgrade dry run resulted in errors {\"package_policy_upgrade\":{\"package_name\":\"crowdstrike\",\"current_version\":\"1.2.3\",\"new_version\":\"1.3.2\",\"status\":\"failure\",\"error\":[{\"key\":\"inputs.crowdstrike_fdr-aws-s3.streams.crowdstrike.fdr.vars.queue_url\",\"message\":[\"Queue URL is required\"]}],\"dryRun\":true}}"
}
This reverts commit 46cff5c.
I tested with 8.3.2 and it works when upgrading from 1.4.1 to 1.5.0 (this version).

With 7.17.6-SNAPHOT it fails during the policy upgrade with

    Error: Root value is not flatten-able, received undefined
@andrewkroh andrewkroh force-pushed the crowdstrike/feature/policy-templates branch from 449af55 to ad5807b Compare July 20, 2022 18:21
@andrewkroh
Copy link
Member Author

This no longer fails on 8.3.2, but it still failing on 7.17.6-SNAPSHOT with the same error.

@elasticmachine
Copy link

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (0/0) 💚
Files 100.0% (0/0) 💚
Classes 100.0% (0/0) 💚
Methods 100.0% (2/2) 💚
Lines 100.0% (0/0) 💚
Conditionals 100.0% (0/0) 💚

@andrewkroh
Copy link
Member Author

Failing tests are blocked on elastic/elastic-package#904.

@botelastic
Copy link

botelastic bot commented Sep 19, 2022

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

@botelastic botelastic bot added the Stalled label Sep 19, 2022
@botelastic
Copy link

botelastic bot commented Oct 19, 2022

Hi! This PR has been stale for a while and we're going to close it as part of our cleanup procedure. We appreciate your contribution and would like to apologize if we have not been able to review it, due to the current heavy load of the team. Feel free to re-open this PR if you think it should stay open and is worth rebasing. Thank you for your contribution!

@botelastic botelastic bot closed this Oct 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[crowdstrike/fdr] logfile input not compatible with falcon_data_replicator.py
3 participants