Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature findings api enhancements #914

Merged
merged 32 commits into from
Mar 13, 2024
Merged

Conversation

riysaxen-amzn
Copy link
Collaborator

@riysaxen-amzn riysaxen-amzn commented Mar 13, 2024

Description

This PR is related to paginations enhancements, for more info please refer to the below github issue

Issues Resolved

795]

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

riysaxen-amzn and others added 16 commits January 15, 2024 23:31
Signed-off-by: Riya Saxena <[email protected]>
@riysaxen-amzn riysaxen-amzn changed the title Feature findings ap ienhancements Feature findings api enhancements Mar 13, 2024
sbcd90
sbcd90 previously approved these changes Mar 13, 2024
}
searchSourceBuilder.fetchSource(true);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we dont pass size, it defaults to 10

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

won't consider more than 10 detector's findings

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, make sense. I had this earlier, i removed it in mid while developing this. Added back

riysaxen-amzn and others added 13 commits March 13, 2024 09:32
Signed-off-by: Riya Saxena <[email protected]>
Signed-off-by: Subhobrata Dey <[email protected]>
Signed-off-by: Riya Saxena <[email protected]>
Signed-off-by: Riya Saxena <[email protected]>
Signed-off-by: Riya Saxena <[email protected]>
Signed-off-by: Riya Saxena <[email protected]>
Signed-off-by: Riya Saxena <[email protected]>
Signed-off-by: Riya Saxena <[email protected]>
Signed-off-by: Riya Saxena <[email protected]>
Signed-off-by: Riya Saxena <[email protected]>
eirsep
eirsep previously approved these changes Mar 13, 2024
@@ -22,6 +23,7 @@
import org.opensearch.commons.alerting.model.FindingWithDocs;
import org.opensearch.commons.alerting.model.Table;
import org.opensearch.core.rest.RestStatus;
import org.opensearch.index.query.*;
import org.opensearch.securityanalytics.action.FindingDto;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove * imports

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed

Signed-off-by: Riya Saxena <[email protected]>
@riysaxen-amzn riysaxen-amzn merged commit 9b59f61 into main Mar 13, 2024
10 of 30 checks passed
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.x 2.x
# Navigate to the new working tree
cd .worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-914-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 9b59f6153956aa86bb6fdf30b9858e27ccd9f849
# Push it to GitHub
git push --set-upstream origin backport/backport-914-to-2.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-914-to-2.x.

riysaxen-amzn added a commit to riysaxen-amzn/security-analytics that referenced this pull request Mar 14, 2024
* get all findings as part of findings API enhancement

Signed-off-by: Riya Saxena <[email protected]>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <[email protected]>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <[email protected]>

* added support for  param in Finding API

Signed-off-by: Riya Saxena <[email protected]>

* added detectionType as param for Findings API enhancements

Signed-off-by: Riya Saxena <[email protected]>

* added few tests to validate findings by params

Signed-off-by: Riya Saxena <[email protected]>

* added test for searchString param in FindingsAPI

Signed-off-by: Riya Saxena <[email protected]>

* adding addiional params findingIds, startTime and endTime as findings API enhancement

Signed-off-by: Riya Saxena <[email protected]>

* added params in getFindingsByDetectorId func

* changed the startTime and endTime req input format

* fix merge conflixt

* fix integ test failures in findings API

* fix integ tests

* refactored the logic

Signed-off-by: Riya Saxena <[email protected]>

* remove unused imports

* address the pr comments

Signed-off-by: Riya Saxena <[email protected]>

* address pr comments

Signed-off-by: Riya Saxena <[email protected]>

* SA integ tests fix

* SA integ tests fix

* fix integ tests for findings

Signed-off-by: Subhobrata Dey <[email protected]>

* fix conflixt errors

Signed-off-by: Riya Saxena <[email protected]>

* fix conflixt errors

Signed-off-by: Riya Saxena <[email protected]>

* fix conflixt errors

Signed-off-by: Riya Saxena <[email protected]>

* fix conflixt errors

Signed-off-by: Riya Saxena <[email protected]>

* fix integ tests

Signed-off-by: Riya Saxena <[email protected]>

* fix integ tests

Signed-off-by: Riya Saxena <[email protected]>

* fix integ tests

Signed-off-by: Riya Saxena <[email protected]>

* fix flaky integ tests

Signed-off-by: Riya Saxena <[email protected]>

* address pr comments

Signed-off-by: Riya Saxena <[email protected]>

---------

Signed-off-by: Riya Saxena <[email protected]>
Signed-off-by: Riya <[email protected]>
Signed-off-by: Subhobrata Dey <[email protected]>
Co-authored-by: Subhobrata Dey <[email protected]>
(cherry picked from commit 9b59f61)
jowg-amazon pushed a commit to jowg-amazon/security-analytics that referenced this pull request Mar 14, 2024
* get all findings as part of findings API enhancement

Signed-off-by: Riya Saxena <[email protected]>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <[email protected]>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <[email protected]>

* added support for  param in Finding API

Signed-off-by: Riya Saxena <[email protected]>

* added detectionType as param for Findings API enhancements

Signed-off-by: Riya Saxena <[email protected]>

* added few tests to validate findings by params

Signed-off-by: Riya Saxena <[email protected]>

* added test for searchString param in FindingsAPI

Signed-off-by: Riya Saxena <[email protected]>

* adding addiional params findingIds, startTime and endTime as findings API enhancement

Signed-off-by: Riya Saxena <[email protected]>

* added params in getFindingsByDetectorId func

* changed the startTime and endTime req input format

* fix merge conflixt

* fix integ test failures in findings API

* fix integ tests

* refactored the logic

Signed-off-by: Riya Saxena <[email protected]>

* remove unused imports

* address the pr comments

Signed-off-by: Riya Saxena <[email protected]>

* address pr comments

Signed-off-by: Riya Saxena <[email protected]>

* SA integ tests fix

* SA integ tests fix

* fix integ tests for findings

Signed-off-by: Subhobrata Dey <[email protected]>

* fix conflixt errors

Signed-off-by: Riya Saxena <[email protected]>

* fix conflixt errors

Signed-off-by: Riya Saxena <[email protected]>

* fix conflixt errors

Signed-off-by: Riya Saxena <[email protected]>

* fix conflixt errors

Signed-off-by: Riya Saxena <[email protected]>

* fix integ tests

Signed-off-by: Riya Saxena <[email protected]>

* fix integ tests

Signed-off-by: Riya Saxena <[email protected]>

* fix integ tests

Signed-off-by: Riya Saxena <[email protected]>

* fix flaky integ tests

Signed-off-by: Riya Saxena <[email protected]>

* address pr comments

Signed-off-by: Riya Saxena <[email protected]>

---------

Signed-off-by: Riya Saxena <[email protected]>
Signed-off-by: Riya <[email protected]>
Signed-off-by: Subhobrata Dey <[email protected]>
Co-authored-by: Subhobrata Dey <[email protected]>
jowg-amazon added a commit that referenced this pull request Mar 15, 2024
* get all findings as part of findings API enhancement (#803)

* get all findings as part of findings API enhancement

Signed-off-by: Riya Saxena <[email protected]>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <[email protected]>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <[email protected]>

* added support for  param in Finding API

Signed-off-by: Riya Saxena <[email protected]>

* added detectionType as param for Findings API enhancements

Signed-off-by: Riya Saxena <[email protected]>

* added few tests to validate findings by params

Signed-off-by: Riya Saxena <[email protected]>

* added test for searchString param in FindingsAPI

Signed-off-by: Riya Saxena <[email protected]>

* adding addiional params findingIds, startTime and endTime as findings API enhancement

Signed-off-by: Riya Saxena <[email protected]>

* added params in getFindingsByDetectorId func

* changed the startTime and endTime req input format

* fix merge conflixt

* fix integ test failures in findings API

* fix integ tests

* fix integ tests for findings

Signed-off-by: Subhobrata Dey <[email protected]>

---------

Signed-off-by: Riya Saxena <[email protected]>
Signed-off-by: Riya <[email protected]>
Signed-off-by: Subhobrata Dey <[email protected]>
Co-authored-by: Subhobrata Dey <[email protected]>

* fix integ test (#918)

Signed-off-by: Joanne Wang <[email protected]>

* Feature findings api enhancements (#914)

* get all findings as part of findings API enhancement

Signed-off-by: Riya Saxena <[email protected]>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <[email protected]>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <[email protected]>

* added support for  param in Finding API

Signed-off-by: Riya Saxena <[email protected]>

* added detectionType as param for Findings API enhancements

Signed-off-by: Riya Saxena <[email protected]>

* added few tests to validate findings by params

Signed-off-by: Riya Saxena <[email protected]>

* added test for searchString param in FindingsAPI

Signed-off-by: Riya Saxena <[email protected]>

* adding addiional params findingIds, startTime and endTime as findings API enhancement

Signed-off-by: Riya Saxena <[email protected]>

* added params in getFindingsByDetectorId func

* changed the startTime and endTime req input format

* fix merge conflixt

* fix integ test failures in findings API

* fix integ tests

* refactored the logic

Signed-off-by: Riya Saxena <[email protected]>

* remove unused imports

* address the pr comments

Signed-off-by: Riya Saxena <[email protected]>

* address pr comments

Signed-off-by: Riya Saxena <[email protected]>

* SA integ tests fix

* SA integ tests fix

* fix integ tests for findings

Signed-off-by: Subhobrata Dey <[email protected]>

* fix conflixt errors

Signed-off-by: Riya Saxena <[email protected]>

* fix conflixt errors

Signed-off-by: Riya Saxena <[email protected]>

* fix conflixt errors

Signed-off-by: Riya Saxena <[email protected]>

* fix conflixt errors

Signed-off-by: Riya Saxena <[email protected]>

* fix integ tests

Signed-off-by: Riya Saxena <[email protected]>

* fix integ tests

Signed-off-by: Riya Saxena <[email protected]>

* fix integ tests

Signed-off-by: Riya Saxena <[email protected]>

* fix flaky integ tests

Signed-off-by: Riya Saxena <[email protected]>

* address pr comments

Signed-off-by: Riya Saxena <[email protected]>

---------

Signed-off-by: Riya Saxena <[email protected]>
Signed-off-by: Riya <[email protected]>
Signed-off-by: Subhobrata Dey <[email protected]>
Co-authored-by: Subhobrata Dey <[email protected]>

---------

Signed-off-by: Riya Saxena <[email protected]>
Signed-off-by: Riya <[email protected]>
Signed-off-by: Subhobrata Dey <[email protected]>
Signed-off-by: Joanne Wang <[email protected]>
Co-authored-by: Riya <[email protected]>
Co-authored-by: Subhobrata Dey <[email protected]>
jowg-amazon pushed a commit to jowg-amazon/security-analytics that referenced this pull request Mar 15, 2024
* get all findings as part of findings API enhancement

Signed-off-by: Riya Saxena <[email protected]>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <[email protected]>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <[email protected]>

* added support for  param in Finding API

Signed-off-by: Riya Saxena <[email protected]>

* added detectionType as param for Findings API enhancements

Signed-off-by: Riya Saxena <[email protected]>

* added few tests to validate findings by params

Signed-off-by: Riya Saxena <[email protected]>

* added test for searchString param in FindingsAPI

Signed-off-by: Riya Saxena <[email protected]>

* adding addiional params findingIds, startTime and endTime as findings API enhancement

Signed-off-by: Riya Saxena <[email protected]>

* added params in getFindingsByDetectorId func

* changed the startTime and endTime req input format

* fix merge conflixt

* fix integ test failures in findings API

* fix integ tests

* refactored the logic

Signed-off-by: Riya Saxena <[email protected]>

* remove unused imports

* address the pr comments

Signed-off-by: Riya Saxena <[email protected]>

* address pr comments

Signed-off-by: Riya Saxena <[email protected]>

* SA integ tests fix

* SA integ tests fix

* fix integ tests for findings

Signed-off-by: Subhobrata Dey <[email protected]>

* fix conflixt errors

Signed-off-by: Riya Saxena <[email protected]>

* fix conflixt errors

Signed-off-by: Riya Saxena <[email protected]>

* fix conflixt errors

Signed-off-by: Riya Saxena <[email protected]>

* fix conflixt errors

Signed-off-by: Riya Saxena <[email protected]>

* fix integ tests

Signed-off-by: Riya Saxena <[email protected]>

* fix integ tests

Signed-off-by: Riya Saxena <[email protected]>

* fix integ tests

Signed-off-by: Riya Saxena <[email protected]>

* fix flaky integ tests

Signed-off-by: Riya Saxena <[email protected]>

* address pr comments

Signed-off-by: Riya Saxena <[email protected]>

---------

Signed-off-by: Riya Saxena <[email protected]>
Signed-off-by: Riya <[email protected]>
Signed-off-by: Subhobrata Dey <[email protected]>
Co-authored-by: Subhobrata Dey <[email protected]>
jowg-amazon added a commit that referenced this pull request Mar 15, 2024
* support object fields in aggregation based sigma rules (#789)

Signed-off-by: Subhobrata Dey <[email protected]>

* Fix duplicate ecs mappings which returns incorrect log index field in mapping view API (#786) (#788)

* field mapping changes

Signed-off-by: Joanne Wang <[email protected]>

* add integ test

Signed-off-by: Joanne Wang <[email protected]>

* turn unmappedfieldaliases as set and add integ test

Signed-off-by: Joanne Wang <[email protected]>

* add comments

Signed-off-by: Joanne Wang <[email protected]>

* fix integ tests

Signed-off-by: Joanne Wang <[email protected]>

* moved logic to method for better readability

Signed-off-by: Joanne Wang <[email protected]>

---------

Signed-off-by: Joanne Wang <[email protected]>

* Fix get mappings view API incorrectly returning ecs path (#867)

* add logic and integ tests to not add duplicate to log-types-config index

Signed-off-by: Joanne Wang <[email protected]>

* change naming for existingFieldMapping and change contains to equals

Signed-off-by: Joanne Wang <[email protected]>

---------

Signed-off-by: Joanne Wang <[email protected]>

* fix integ test (#918)

Signed-off-by: Joanne Wang <[email protected]>

* get all findings as part of findings API enhancement (#803)

* get all findings as part of findings API enhancement

Signed-off-by: Riya Saxena <[email protected]>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <[email protected]>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <[email protected]>

* added support for  param in Finding API

Signed-off-by: Riya Saxena <[email protected]>

* added detectionType as param for Findings API enhancements

Signed-off-by: Riya Saxena <[email protected]>

* added few tests to validate findings by params

Signed-off-by: Riya Saxena <[email protected]>

* added test for searchString param in FindingsAPI

Signed-off-by: Riya Saxena <[email protected]>

* adding addiional params findingIds, startTime and endTime as findings API enhancement

Signed-off-by: Riya Saxena <[email protected]>

* added params in getFindingsByDetectorId func

* changed the startTime and endTime req input format

* fix merge conflixt

* fix integ test failures in findings API

* fix integ tests

* fix integ tests for findings

Signed-off-by: Subhobrata Dey <[email protected]>

---------

Signed-off-by: Riya Saxena <[email protected]>
Signed-off-by: Riya <[email protected]>
Signed-off-by: Subhobrata Dey <[email protected]>
Co-authored-by: Subhobrata Dey <[email protected]>

* Feature findings api enhancements (#914)

* get all findings as part of findings API enhancement

Signed-off-by: Riya Saxena <[email protected]>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <[email protected]>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <[email protected]>

* added support for  param in Finding API

Signed-off-by: Riya Saxena <[email protected]>

* added detectionType as param for Findings API enhancements

Signed-off-by: Riya Saxena <[email protected]>

* added few tests to validate findings by params

Signed-off-by: Riya Saxena <[email protected]>

* added test for searchString param in FindingsAPI

Signed-off-by: Riya Saxena <[email protected]>

* adding addiional params findingIds, startTime and endTime as findings API enhancement

Signed-off-by: Riya Saxena <[email protected]>

* added params in getFindingsByDetectorId func

* changed the startTime and endTime req input format

* fix merge conflixt

* fix integ test failures in findings API

* fix integ tests

* refactored the logic

Signed-off-by: Riya Saxena <[email protected]>

* remove unused imports

* address the pr comments

Signed-off-by: Riya Saxena <[email protected]>

* address pr comments

Signed-off-by: Riya Saxena <[email protected]>

* SA integ tests fix

* SA integ tests fix

* fix integ tests for findings

Signed-off-by: Subhobrata Dey <[email protected]>

* fix conflixt errors

Signed-off-by: Riya Saxena <[email protected]>

* fix conflixt errors

Signed-off-by: Riya Saxena <[email protected]>

* fix conflixt errors

Signed-off-by: Riya Saxena <[email protected]>

* fix conflixt errors

Signed-off-by: Riya Saxena <[email protected]>

* fix integ tests

Signed-off-by: Riya Saxena <[email protected]>

* fix integ tests

Signed-off-by: Riya Saxena <[email protected]>

* fix integ tests

Signed-off-by: Riya Saxena <[email protected]>

* fix flaky integ tests

Signed-off-by: Riya Saxena <[email protected]>

* address pr comments

Signed-off-by: Riya Saxena <[email protected]>

---------

Signed-off-by: Riya Saxena <[email protected]>
Signed-off-by: Riya <[email protected]>
Signed-off-by: Subhobrata Dey <[email protected]>
Co-authored-by: Subhobrata Dey <[email protected]>

* fix findings api integ tests

Signed-off-by: Joanne Wang <[email protected]>

---------

Signed-off-by: Subhobrata Dey <[email protected]>
Signed-off-by: Joanne Wang <[email protected]>
Signed-off-by: Riya Saxena <[email protected]>
Signed-off-by: Riya <[email protected]>
Co-authored-by: Subhobrata Dey <[email protected]>
Co-authored-by: Riya <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants