[Fleet] Change agentPolicyUpdateEventHandler to use soClient from appContext when Fleet Policy Change Action's #79341

paul-tavares · 2020-10-02T18:46:20Z

Summary

Change agentPolicyUpdateEventHandler to use soClient from appContext when Fleet Policy Change Action's instead of the soClient passed on input, since that soClient could have been one from a non-Fleet plugin application (ex. Endpoint).
This changes fixes a problem where a decrypt error was being logged and policy change actions not created after the security solution manifest management updated a Fleet Endpoint Integration policy.

To recreate the original problem, the following steps were followed:

Bring up Fleet, add Agent with Endpoint
Add a trusted app from the trusted apps UI in Security Solution (select Path for field)
Wait for the log entry to appear indicating a new version of the manifest was committed. Ex:

[info][plugins][plugins][securitySolution][securitySolution] Committed manifest 1.0.1

See the error logs shown below in the console
Agent Policy never leaves “out-of-date” state

Original Error Seen was:

server    log   [13:58:32.897] [info][plugins][plugins][securitySolution][securitySolution] Committed manifest 1.0.1
server    log   [13:58:36.969] [info][plugins][plugins][securitySolution][securitySolution] Cleaned up artifact endpoint-trustlist-windows-v1-d801aa1fb7ddcc330a5e3173372ea6af4a3d08ec58074478e85aa5603e926658
server    log   [13:58:39.668] [error][encryptedSavedObjects][plugins] Failed to decrypt "data" attribute: Unsupported state or unable to authenticate data
server    log   [13:58:39.670] [error][ingestManager][plugins] { Error: Unable to decrypt attribute "data"
    at EncryptedSavedObjectsService.attributesToDecryptIterator (/Users/klogan/kibana-kev/kibana/x-pack/plugins/encrypted_saved_objects/server/crypto/encrypted_saved_objects_service.ts:407:15)
    at attributesToDecryptIterator.throw (<anonymous>)
    at EncryptedSavedObjectsService.decryptAttributes (/Users/klogan/kibana-kev/kibana/x-pack/plugins/encrypted_saved_objects/server/crypto/encrypted_saved_objects_service.ts:336:9)
  attributeName: 'data',
  operation: 1,
  cause:
   Error: Unsupported state or unable to authenticate data
       at Decipheriv.final (internal/crypto/cipher.js:164:28)
       at decrypt (/Users/klogan/kibana-kev/kibana/x-pack/node_modules/@elastic/node-crypto/src/crypto.ts:157:73)
       at Object.<anonymous> (/Users/klogan/kibana-kev/kibana/x-pack/node_modules/@elastic/node-crypto/src/crypto.ts:188:14)
       at step (/Users/klogan/kibana-kev/kibana/x-pack/node_modules/@elastic/node-crypto/lib/crypto.js:45:23)
       at Object.next (/Users/klogan/kibana-kev/kibana/x-pack/node_modules/@elastic/node-crypto/lib/crypto.js:26:53)
       at fulfilled (/Users/klogan/kibana-kev/kibana/x-pack/node_modules/@elastic/node-crypto/lib/crypto.js:17:58) }
server   error  [13:57:33.840]  Error: Internal Server Error
    at HapiResponseAdapter.toError (/Users/klogan/kibana-kev/kibana/src/core/server/http/router/response_adapter.ts:129:19)
    at HapiResponseAdapter.toHapiResponse (/Users/klogan/kibana-kev/kibana/src/core/server/http/router/response_adapter.ts:79:19)
    at HapiResponseAdapter.handle (/Users/klogan/kibana-kev/kibana/src/core/server/http/router/response_adapter.ts:74:17)
    at Router.handle (/Users/klogan/kibana-kev/kibana/src/core/server/http/router/router.ts:273:34)

…reation of actions

elasticmachine · 2020-10-02T18:46:23Z

Pinging @elastic/ingest-management (Team:Ingest Management)

nchaulet · 2020-10-02T18:51:58Z

The change look good to me, it's probably worth investigating, if endpoint should use a SO client with encrypted saved object plugin, also there is probably some other plugins that are missing like the security one

kevinlog · 2020-10-02T18:59:11Z

x-pack/plugins/ingest_manager/server/services/agent_policy_update.ts

  // If no admin user and no default output fleet is not enabled just skip this hook
  if (!adminUser || !outputId) {
    return;
  }

+  // `soClient` from ingest `appContextService` is used to create policy change actions
+  // to ensure encrypted SOs are handled correctly
+  const internalSoClient = appContextService.getInternalUserSOClient(fakeRequest);


the alternative to this would be to include the encrypted SO plugin in Security_Solution?

@neptunian FYI, is this what would be needed by the error you were seeing? I think it had to do with api key generation?

@kevinlog yes, but I don't know much about how that could/would impact the creation of the artifacts as well as the Exceptions List since the current soClient seems to be passed on to those other areas as well as Manifest manger. Really need @madirey (whenever she has some time) to comment on this.

Another alternative might be to use a encryptedSavedObject client only for manifest manager, since I think that is the only one that uses the Ingest policy services. I looked at how Ingest creates their soClient and I think we have what is needed in security solution plugin.start() to create it.

kevinlog · 2020-10-02T21:10:46Z

@elasticmachine merge upstream

kevinlog · 2020-10-03T13:56:32Z

@elasticmachine merge upstream

kevinlog · 2020-10-03T14:39:43Z

@elasticmachine merge upstream

kibanamachine · 2020-10-03T16:11:27Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 46c9c82

Metrics [docs]

✅ unchanged

History

💔 Build #79345 failed ff8a4f3
💔 Build #79289 failed ea9747c

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

…reation of actions (#79341) (#79393) Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Paul Tavares <[email protected]> Co-authored-by: Kibana Machine <[email protected]>

* master: (128 commits) add core-js production dependency (elastic#79395) Add support for sharing saved objects to all spaces (elastic#76132) [Alerting UI] Display a banner to users when some alerts have failures, added alert statuses column and filters (elastic#79038) load js-yaml lazily (elastic#79092) skip flaky suite (elastic#77278) Fix agentPolicyUpdateEventHandler() to use app context soClient for creation of actions (elastic#79341) [Security Solution] Untitled Timeline created when first action is to add note (elastic#78988) [Security Solutions][Detection Engine] Updates the edit rules page to:wq! only have what is selected for editing (elastic#79233) Cleanup yarn.lock from duplicates (elastic#66617) [kbn/optimizer] implement more efficient auto transpilation for node (elastic#79052) [Ingest Manager] Rename Fleet setup and requirement, Fleet => Central… (elastic#79291) [core/server/plugins] don't run discovery in dev server parent process (take 2) (elastic#79358) [babel/register] remove from build (take 2) (elastic#79379) [Security Solution] Changes rules table tag display (elastic#77102) define integrationTestRoot in config file and use to define screensho… (elastic#79247) Revert "[babel/register] remove from build (elastic#79176)" skip flaky suite (elastic#75241) [Uptime] Synthetics UI (elastic#77960) [Security Solution] [Detections] Only display actions options if user has "read" privileges (elastic#78812) [babel/register] remove from build (elastic#79176) ...

* master: (288 commits) add core-js production dependency (elastic#79395) Add support for sharing saved objects to all spaces (elastic#76132) [Alerting UI] Display a banner to users when some alerts have failures, added alert statuses column and filters (elastic#79038) load js-yaml lazily (elastic#79092) skip flaky suite (elastic#77278) Fix agentPolicyUpdateEventHandler() to use app context soClient for creation of actions (elastic#79341) [Security Solution] Untitled Timeline created when first action is to add note (elastic#78988) [Security Solutions][Detection Engine] Updates the edit rules page to only have what is selected for editing (elastic#79233) Cleanup yarn.lock from duplicates (elastic#66617) [kbn/optimizer] implement more efficient auto transpilation for node (elastic#79052) [Ingest Manager] Rename Fleet setup and requirement, Fleet => Central… (elastic#79291) [core/server/plugins] don't run discovery in dev server parent process (take 2) (elastic#79358) [babel/register] remove from build (take 2) (elastic#79379) [Security Solution] Changes rules table tag display (elastic#77102) define integrationTestRoot in config file and use to define screensho… (elastic#79247) Revert "[babel/register] remove from build (elastic#79176)" skip flaky suite (elastic#75241) [Uptime] Synthetics UI (elastic#77960) [Security Solution] [Detections] Only display actions options if user has "read" privileges (elastic#78812) [babel/register] remove from build (elastic#79176) ...

Fix agentPolicyUpdateEventHandler() to use app context soClient for c…

ea9747c

…reation of actions

paul-tavares added v8.0.0 release_note:skip Skip the PR/issue when compiling release notes v7.10.0 Team:Fleet Team label for Observability Data Collection Fleet team labels Oct 2, 2020

paul-tavares requested a review from a team October 2, 2020 18:46

paul-tavares self-assigned this Oct 2, 2020

paul-tavares requested a review from nchaulet October 2, 2020 18:49

nchaulet approved these changes Oct 2, 2020

View reviewed changes

kevinlog approved these changes Oct 2, 2020

View reviewed changes

kevinlog reviewed Oct 2, 2020

View reviewed changes

Merge branch 'master' into fix/ingest-agent-policy-update-change-action

ff8a4f3

Merge branch 'master' into fix/ingest-agent-policy-update-change-action

46c9c82

kevinlog merged commit e859fe7 into elastic:master Oct 3, 2020

kevinlog deleted the fix/ingest-agent-policy-update-change-action branch October 3, 2020 23:24

kevinlog mentioned this pull request Oct 3, 2020

[7.x] Fix agentPolicyUpdateEventHandler() to use app context soClient for creation of actions (#79341) #79393

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Fleet] Change agentPolicyUpdateEventHandler to use soClient from appContext when Fleet Policy Change Action's #79341

[Fleet] Change agentPolicyUpdateEventHandler to use soClient from appContext when Fleet Policy Change Action's #79341

paul-tavares commented Oct 2, 2020

elasticmachine commented Oct 2, 2020

nchaulet commented Oct 2, 2020 •

edited

Loading

kevinlog Oct 2, 2020

paul-tavares Oct 2, 2020

kevinlog commented Oct 2, 2020

kevinlog commented Oct 3, 2020

kevinlog commented Oct 3, 2020

kibanamachine commented Oct 3, 2020

[Fleet] Change agentPolicyUpdateEventHandler to use soClient from appContext when Fleet Policy Change Action's #79341

[Fleet] Change agentPolicyUpdateEventHandler to use soClient from appContext when Fleet Policy Change Action's #79341

Conversation

paul-tavares commented Oct 2, 2020

Summary

To recreate the original problem, the following steps were followed:

Original Error Seen was:

elasticmachine commented Oct 2, 2020

nchaulet commented Oct 2, 2020 • edited Loading

kevinlog Oct 2, 2020

Choose a reason for hiding this comment

paul-tavares Oct 2, 2020

Choose a reason for hiding this comment

kevinlog commented Oct 2, 2020

kevinlog commented Oct 3, 2020

kevinlog commented Oct 3, 2020

kibanamachine commented Oct 3, 2020

💚 Build Succeeded

Metrics [docs]

History

nchaulet commented Oct 2, 2020 •

edited

Loading