Actions add proxy support #74289

YulNaumenko · 2020-08-04T18:14:10Z

Added proxy settings for defining proxyUrl and proxyHeaders under kibana.yml xpack.actions configuration.
The decision was to use HTTP/HTTPS proxy, not SOCKS, because it is the most commonly used and all action types libraries support it.
For enabling self-signed certificates added rejectUnauthorized: false for all external requests (axios, nodemailer, slack).

For email action type is used Nodemailer library and proxy configuration is built-in https://nodemailer.com/smtp/proxies/#1-using-http-proxy
For slack action type is used node-slack-sdk, which does not have a built-in proxy, but allows to configure it using https-proxy-agent. Under the hood node-slack-sdk/webhook implemented by using axios library. So approach here is really the same - used HttpsProxyAgent and HttpProxyAgent.
For the rest of action types axios library is used to make requests to external services. This library has built-in proxy configuration for HTTPS, but for enabling self-signed certificates we need to setting rejectUnauthorized: false which is configurable only under the agent options. So implemented proxy support using httpsAgent for HTTPS proxy and HttpProxyAgent for the HTTP proxy.

Resolve #50267

elasticmachine · 2020-08-04T18:14:13Z

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

…-support # Conflicts: # x-pack/plugins/actions/server/builtin_action_types/slack.ts # x-pack/plugins/actions/server/builtin_action_types/webhook.test.ts # x-pack/plugins/actions/server/types.ts

…-support # Conflicts: # x-pack/package.json

…-support # Please enter a commit message to explain why this merge is necessary, # especially if it merges an updated upstream into a topic branch. # # Lines starting with '#' will be ignored, and an empty message aborts # the commit.

mikecote

I'm just about half way through the code, I figured I'd send what I have so far as I won't have time until tomorrow to finish the rest.

I think there should also be a mention of proxy in the developing new action types doc.

x-pack/plugins/actions/server/types.ts

x-pack/plugins/actions/server/lib/action_executor.ts

x-pack/plugins/actions/server/plugin.ts

x-pack/plugins/actions/server/builtin_action_types/slack.ts

x-pack/plugins/actions/server/builtin_action_types/resilient/index.ts

x-pack/plugins/actions/server/builtin_action_types/lib/get_proxy_agent.ts

pmuellr

LGTM - I made a bunch of comments, but many are just comments or nits, and I don't think there are any blockers. I'm guessing we may need to tweak the proxy bits over time - my experience is that I've had to do that whenever I've added proxy support like this to a product - so anything I mentioned that would be "nice to have", we can do later as needed.

pmuellr · 2020-08-12T22:16:58Z

x-pack/plugins/actions/server/builtin_action_types/lib/send_email.ts

@@ -6,10 +6,10 @@

 // info on nodemailer: https://nodemailer.com/about/
 import nodemailer from 'nodemailer';
-


I was confused by the references to Nodemailer and using it with an http proxy - I hadn't realized that was even possible! Quick check yields thread - 2nd entry there is very good talking about CONNECT, etc - http://squid-web-proxy-cache.1019090.n4.nabble.com/Can-squid-be-configured-as-SMTP-SMTPS-proxy-td2727188.html

pmuellr · 2020-08-13T14:59:04Z

x-pack/package.json

@@ -72,8 +72,9 @@
    "@types/gulp": "^4.0.6",
    "@types/hapi__wreck": "^15.0.1",
    "@types/he": "^1.1.1",
-    "@types/hoist-non-react-statics": "^3.3.1",


seems slightly odd that the package names got resorted here, though it looks like they sorted CORRECTLY now, and weren't before. Like someone previously edited the file manually. It's just @types/ packages anyway though, so not a big deal, and it looks fine to me.

x-pack/plugins/actions/server/builtin_action_types/index.ts

x-pack/plugins/actions/server/builtin_action_types/lib/get_proxy_agent.ts

x-pack/plugins/actions/server/plugin.ts

x-pack/plugins/actions/server/types.ts

x-pack/test/alerting_api_integration/common/config.ts

x-pack/test/alerting_api_integration/common/lib/get_proxy_server.ts

...test/alerting_api_integration/security_and_spaces/tests/actions/builtin_action_types/jira.ts

mikecote

Changes LGTM, great work! I've finished reviewing the remaining files.

mikecote · 2020-08-14T12:58:50Z

x-pack/plugins/actions/server/builtin_action_types/case/types.ts

+  createExternalService: (
+    credentials: ExternalServiceCredentials,
+    logger: Logger,
+    proxySettings?: any


Can proxySettings use ProxySettings type instead of using any?

I've a got in this case node plugins/security_solution/scripts/check_circular_deps failing. I made it 'any', because when the Case team change their Jira, IBM resilient according to the ServiceNow example this code will be removed.

kibanamachine · 2020-08-14T21:12:51Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: f669e08

Build metrics

✅ unchanged

History

💚 Build #68077 succeeded 83be05c
💔 Build #68072 failed b043eeb
💔 Build #68066 failed bb9fcac
💔 Build #68011 failed 2dacff0
💔 Build #67862 failed d7a7254

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

* Added proxy support for action types * Fixed tests * added rejectUnauthorizedCertificates config setting * removed slack not used code * Fixed Slack proxy * fixed typecheck errors * Cleanup code * Fixed slack * Added unit tests * added proxy server for test * Fixed build * Added functional tests * fixed due to comments * Fixed tests and some changes due to comments * Fixed functional tests * fixed circular deps * Added proxy unit test to action type

* master: (24 commits) [ML] Functional tests - skip regression and classification tests [Ingest Manager] fix removing ingest pipelines from elasticsearch (elastic#75092) move tests for placeholder indices to setup (elastic#75096) [jest] temporarily extend default test timeout (elastic#75118) [cli] remove reference to removed --optimize flag (elastic#75083) skip flaky suite (elastic#75044) Adding /etc/rc.d/init.d/functions to the init script when present to … (elastic#22985) [jenkins] add pipeline for hourly security solution cypress tests (elastic#75087) [Reporting/Flaky Test] Skip test for paging list of reports (elastic#75075) remove .kbn-optimizer-cache upload (elastic#75086) skip flaky suite (elastic#74814) Actions add proxy support (elastic#74289) [ILM] TS conversion of Edit policy components (elastic#74747) [Resolver] simulator tests select elements directly instead of using descendant selectors. (elastic#75058) [Enterprise Search] Add Workplace Search side navigation (elastic#74894) [Security solution] Sourcerer: Kibana index pattern selector for security views (elastic#74706) [Logs UI] Remove apollo deps from log link-to routes (elastic#74502) [Maps] add map configurations to docker list (elastic#75035) [functional test][saved objects] update tests for additional copy saved objects to space (elastic#74907) Make the alerts plugin support generics (elastic#72716) ...

* Added proxy support for action types * Fixed tests * added rejectUnauthorizedCertificates config setting * removed slack not used code * Fixed Slack proxy * fixed typecheck errors * Cleanup code * Fixed slack * Added unit tests * added proxy server for test * Fixed build * Added functional tests * fixed due to comments * Fixed tests and some changes due to comments * Fixed functional tests * fixed circular deps * Added proxy unit test to action type

YulNaumenko added 3 commits August 4, 2020 11:07

Added proxy support for action types

499cbbd

Fixed tests

229ae77

added rejectUnauthorizedCertificates config setting

88ef76c

YulNaumenko added Feature:Alerting v8.0.0 release_note:skip Skip the PR/issue when compiling release notes Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) labels Aug 4, 2020

YulNaumenko requested a review from a team as a code owner August 4, 2020 18:14

YulNaumenko self-assigned this Aug 4, 2020

YulNaumenko added 9 commits August 4, 2020 16:08

removed slack not used code

cf98dd3

Fixed Slack proxy

ebb99a4

Merge remote-tracking branch 'upstream/master' into actions-add-proxy…

cbcbfc6

…-support # Conflicts: # x-pack/plugins/actions/server/builtin_action_types/slack.ts # x-pack/plugins/actions/server/builtin_action_types/webhook.test.ts # x-pack/plugins/actions/server/types.ts

fixed typecheck errors

e2d8bf0

Cleanup code

b8cbda1

Fixed slack

e4debd0

Merge remote-tracking branch 'upstream/master' into actions-add-proxy…

4439096

…-support # Conflicts: # x-pack/package.json

Added unit tests

9db7be1

added proxy server for test

4895991

mikecote self-requested a review August 13, 2020 00:06

YulNaumenko added 3 commits August 12, 2020 17:30

Fixed build

42639a3

Added functional tests

d7a7254

mikecote reviewed Aug 13, 2020

View reviewed changes

pmuellr approved these changes Aug 13, 2020

View reviewed changes

YulNaumenko added 4 commits August 13, 2020 12:03

fixed due to comments

2dacff0

Fixed tests and some changes due to comments

bb9fcac

Fixed functional tests

b043eeb

fixed circular deps

83be05c

mikecote approved these changes Aug 14, 2020

View reviewed changes

Added proxy unit test to action type

f669e08

YulNaumenko merged commit 52bd6d9 into elastic:master Aug 14, 2020

YulNaumenko mentioned this pull request Aug 14, 2020

[7.x] Actions add proxy support (#74289) #75091

Merged

tomrade mentioned this pull request Aug 20, 2020

[Ingest Manager] Kibana does not take into account proxy for package download #70710

Closed

YulNaumenko added v7.10.0 v7.9.3 labels Sep 30, 2020

mikecote added v7.9.0 release_note:enhancement and removed v7.10.0 v7.9.3 release_note:skip Skip the PR/issue when compiling release notes labels Sep 30, 2020

KOTungseth added a commit to KOTungseth/kibana that referenced this pull request Sep 30, 2020

[DOCS] Adds elastic#74289 to 7.9 Release Notes

bc58452

KOTungseth mentioned this pull request Sep 30, 2020

[DOCS] Adds #74289 to 7.9 Release Notes #78980

Merged

1 task

KOTungseth added a commit that referenced this pull request Sep 30, 2020

[DOCS] Adds #74289 to 7.9 Release Notes (#78980)

3f16b74

KOTungseth added a commit to KOTungseth/kibana that referenced this pull request Sep 30, 2020

[DOCS] Adds elastic#74289 to 7.9 Release Notes

3ad1b06

KOTungseth mentioned this pull request Sep 30, 2020

[7.x] [DOCS] Adds #74289 to 7.9 Release Notes (#78980) #78987

Merged

KOTungseth added a commit that referenced this pull request Oct 1, 2020

[DOCS] Adds #74289 to 7.9 Release Notes (#78987)

554ec3f

gmmorris added v7.10.0 and removed v7.9.0 labels Oct 13, 2020

KOTungseth mentioned this pull request Oct 14, 2020

[DOCS] Removed 74289 from Release Notes #80518

Merged

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Actions add proxy support #74289

Actions add proxy support #74289

YulNaumenko commented Aug 4, 2020 •

edited

Loading

elasticmachine commented Aug 4, 2020

mikecote left a comment •

edited

Loading

pmuellr left a comment

pmuellr Aug 12, 2020

pmuellr Aug 13, 2020

mikecote left a comment

mikecote Aug 14, 2020

YulNaumenko Aug 14, 2020

kibanamachine commented Aug 14, 2020

		@@ -6,10 +6,10 @@

		// info on nodemailer: https://nodemailer.com/about/
		import nodemailer from 'nodemailer';

Actions add proxy support #74289

Actions add proxy support #74289

Conversation

YulNaumenko commented Aug 4, 2020 • edited Loading

elasticmachine commented Aug 4, 2020

mikecote left a comment • edited Loading

Choose a reason for hiding this comment

pmuellr left a comment

Choose a reason for hiding this comment

pmuellr Aug 12, 2020

Choose a reason for hiding this comment

pmuellr Aug 13, 2020

Choose a reason for hiding this comment

mikecote left a comment

Choose a reason for hiding this comment

mikecote Aug 14, 2020

Choose a reason for hiding this comment

YulNaumenko Aug 14, 2020

Choose a reason for hiding this comment

kibanamachine commented Aug 14, 2020

💚 Build Succeeded

Build metrics

History

YulNaumenko commented Aug 4, 2020 •

edited

Loading

mikecote left a comment •

edited

Loading