Attempt to move control of registration for nodes and pipelines to the audit broker #24505
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
the
hashicorp-contributed-pr
|
CI Results: |
|
Build Results: |
kubawi
approved these changes
Dec 13, 2023
peteski22
merged commit e0afe21
into
kubawi/VAULT-22481/audit-filtering-wiring-it-all-up
101 checks passed
peteski22
deleted the
peteski22/VAULT-22481/audit-filtering-strikes-back
branch
peteski22
pushed a commit
that referenced
this pull request
Dec 15, 2023
* Support filter nodes in backend factories and add some tests * More tests and cleanup * Attempt to move control of registration for nodes and pipelines to the audit broker (#24505) * invert control of the pipelines/nodes to the audit broker vs. within each backend * update noop audit test code to implement the pipeliner interface * noop mount path has trailing slash * attempting to make NoopAudit more friendly * NoopAudit uses known salt * Refactor audit.ProcessManual to support filter nodes * HasFiltering * rename the pipeliner * use exported AuditEvent in Filter * Add tests for registering and deregistering backends on the audit broker * Add missing licence header to one file, fix a typo in two tests --------- Co-authored-by: Peter Wilson <[email protected]>
peteski22
pushed a commit
that referenced
this pull request
Dec 18, 2023
* VAULT-22481: Audit filter node (#24465) * Initial commit on adding filter nodes for audit * tests for audit filter * test: longer filter - more conditions * copywrite headers * Check interface for the right type * Add audit filtering feature (#24554) * Support filter nodes in backend factories and add some tests * More tests and cleanup * Attempt to move control of registration for nodes and pipelines to the audit broker (#24505) * invert control of the pipelines/nodes to the audit broker vs. within each backend * update noop audit test code to implement the pipeliner interface * noop mount path has trailing slash * attempting to make NoopAudit more friendly * NoopAudit uses known salt * Refactor audit.ProcessManual to support filter nodes * HasFiltering * rename the pipeliner * use exported AuditEvent in Filter * Add tests for registering and deregistering backends on the audit broker * Add missing licence header to one file, fix a typo in two tests --------- Co-authored-by: Peter Wilson <[email protected]> * Add changelog file * update bexpr datum to use a strong type * go docs updates * test path * PR review comments * handle scenarios/outcomes from broker.send * don't need to re-check the complete sinks * add extra check to deregister to ensure that re-registering non-filtered device sets sink threshold * Ensure that the multierror is appended before attempting to return it --------- Co-authored-by: Peter Wilson <[email protected]>
Monkeychip
pushed a commit
that referenced
this pull request
Jan 7, 2024
* VAULT-22481: Audit filter node (#24465) * Initial commit on adding filter nodes for audit * tests for audit filter * test: longer filter - more conditions * copywrite headers * Check interface for the right type * Add audit filtering feature (#24554) * Support filter nodes in backend factories and add some tests * More tests and cleanup * Attempt to move control of registration for nodes and pipelines to the audit broker (#24505) * invert control of the pipelines/nodes to the audit broker vs. within each backend * update noop audit test code to implement the pipeliner interface * noop mount path has trailing slash * attempting to make NoopAudit more friendly * NoopAudit uses known salt * Refactor audit.ProcessManual to support filter nodes * HasFiltering * rename the pipeliner * use exported AuditEvent in Filter * Add tests for registering and deregistering backends on the audit broker * Add missing licence header to one file, fix a typo in two tests --------- Co-authored-by: Peter Wilson <[email protected]> * Add changelog file * update bexpr datum to use a strong type * go docs updates * test path * PR review comments * handle scenarios/outcomes from broker.send * don't need to re-check the complete sinks * add extra check to deregister to ensure that re-registering non-filtered device sets sink threshold * Ensure that the multierror is appended before attempting to return it --------- Co-authored-by: Peter Wilson <[email protected]>
peteski22
pushed a commit
that referenced
this pull request
Jan 10, 2024
* VAULT-22481: Audit filter node (#24465) * Initial commit on adding filter nodes for audit * tests for audit filter * test: longer filter - more conditions * copywrite headers * Check interface for the right type * Add audit filtering feature (#24554) * Support filter nodes in backend factories and add some tests * More tests and cleanup * Attempt to move control of registration for nodes and pipelines to the audit broker (#24505) * invert control of the pipelines/nodes to the audit broker vs. within each backend * update noop audit test code to implement the pipeliner interface * noop mount path has trailing slash * attempting to make NoopAudit more friendly * NoopAudit uses known salt * Refactor audit.ProcessManual to support filter nodes * HasFiltering * rename the pipeliner * use exported AuditEvent in Filter * Add tests for registering and deregistering backends on the audit broker * Add missing licence header to one file, fix a typo in two tests --------- Co-authored-by: Peter Wilson <[email protected]> * Add changelog file * initial work on global metrics for sink success/failure * initial work to add a fallback device for audit * Return when we have outright errors * Improve comment * Remove unneeded options on NewBroker and remove the policy opts elsewhere * Remove duplicate node registration code * Add more tests for audit backends * ensure we return the multierror as soon as possible, and append it correctly * error tweaks for audit: log req/resp * extract the registration for fallback/normal devices, and ensure we always add to backends when successful * slightly nicer error message rather than returning the raw err * refactor the deregister methods for audit broker * Prevent issues if fallback device is the first device added * Bail early when the user tries adding more than one fallback audit device * Check if there is an existing fallback audit device when setting the required sinks threshold for an audit broker * Use the right ParseBool in audit backends * Tweak the way we check for the threshold to make it clear why we ignore fallback * Ensure all 'fallback' settings look the same * nicer formatting of error * broker tests for Register * Deregister tests * Deregister checks if registered before attempting * Comment improvement * Multiple Deregister calls are OK * Fallback not required in this test * Sanitise input for Deregister * Locking mixup * fix test * Add changelog * Check fallback broker's sink success threshold for register/deregister * Remove changelog * updated * better name for the audit metrics labelers * extra test * remove name from metric counter type * update func calls for NewMetricsCounter * labelers should be pointers to the instance * revert audit_test complaints about the header * use constant value for the metric label on a fallback miss * remove vault prefix from metric labels * US spelling for labeler and adjust the way the labels are returned * Fixed name and type we're testing for * Defensive addition to HasFiltering (no nodemap no filter node) * Remove dupe code block * Revert to using armon/go-metrics * Fallback miss fix * PR feedback updates * consistent format for configure methods * Updated telemetry set up based on PR feedback --------- Co-authored-by: Kuba Wieczorek <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR tries to provide
AuditBrokerwith more information to let it manage pipelines and set the success threshold (for sinks) based on whether we have: