[Remote clusters] Cloud deployment form when adding new cluster #94450

yuliacech · 2021-03-11T15:57:02Z

Summary

Addresses some of #90894
This PR adds a new input for Cloud (both ESS and ECE) users that allows them to configure a remote cluster with only a Cloud endpoint url (no need to fill out proxy address and server name). Also proxy mode that is default for Cloud is being forced in the form (no switch to sniff mode).

Screenshot

Implementation

Remote clusters on Cloud context

only same environment connections are possible
only proxy mode is allowed
default port 9400 (that is not likely to change)
Cloud endpoint url can be used to fill proxy address and server name with following logic:
- Cloud endpoint url = <protocol>://<domain>:<port>
- proxy address = <domain>:9400
- server name = <domain>

Cloud endpoint url input

Cluster	New	Edit
non-Cloud	sniff + proxy mode, no Cloud url input	sniff + proxy mode, no Cloud url input
Cloud	proxy mode, Cloud url input enabled	proxy mode, Cloud url input possibly enabled*

*logic for enabling Cloud url input when editing an existing cluster:

proxy address uses Cloud default port 9400
proxy address without port === server name

Screenshot and GIF

Screenshots

Cloud url input enabled

Cloud url disabled

GIF

Apr-12-2021.17-37-42.mp4

How to test

When running Kibana locally add a value for xpack.cloud.id to kibana.yml file to simulate Cloud environment

xpack.cloud.id: 'remote-deployment:dXMtY2VudHJhbDEuZ2NwLmZvdW5kaXQubm8kODU0NDAwYjA4OTJhNGFhMzk2M2ZhZTg0MzRiOGRhODEkZDVmZTQzZTFmZWM3NDljM2JhNDgyYTNlNTY5YmRjMTc='

Checklist

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
Any UI touched in this PR is usable by keyboard only (learn more about keyboard accessibility)
Any UI touched in this PR does not create any new axe failures (run axe in browser: FF, Chrome)

Release Note

It's now possible to configure a remote cluster on Cloud using only Cloud endpoint url.

elasticmachine · 2021-03-11T16:58:55Z

Pinging @elastic/es-ui (Team:Elasticsearch UI)

zanbel · 2021-03-15T06:22:10Z

Thanks, @yuliacech ! I think this approach works really nicely.

A couple of comments:

I noticed you used the term Elasticsearch endpoint URL in the connection mode description but the field name is Cloud deployment url. Maybe it will be easier for users if you use the same term. We should also use either URL or url.
The term Proxy address and Server name appear with an upper case in the info message but all lower cases in the toggle button.
Following some recent conversions, summarized by @juliocamarero here elastic/cloud#77079, we should be able to correctly configure the remote cluster when using an alias. Not sure if you want to address this scenario as part of this PR and update the info message or in a new one.

yuliacech · 2021-03-16T13:51:38Z

@elasticmachine merge upstream

afharo

Core changes LGTM!

yuliacech · 2021-03-17T08:04:15Z

@elasticmachine merge upstream

mdefazio · 2021-03-18T13:23:28Z

Is there a reason we auto-open the flyout after adding a remote url? Is there a connection check that happens here?
Can we just show a toast indicating it was successfully? With the option to see details?

yuliacech · 2021-03-18T13:48:49Z

@elasticmachine merge upstream

yuliacech · 2021-03-18T14:15:56Z

Hi @mdefazio, the flyout opens automatically because the form redirects to the list with details view after a cluster is added successfully. This was not changed in this work but I'm happy to address your feedback in a separate PR. Should the form still redirect to the list view but display a toast notification instead of the details flyout? Or we could possibly disable the redirection all together, WDYT?

kellyemurphy · 2021-03-31T14:15:24Z

@yuliacech , here's a quick mockup based on our conversation. I would recommend simply including the Elasticsearch logo as opposed to adding a screenshot. This would definitely still need copy review.

@mdefazio The logos in the Cloud UI are going away, in a future project.

As for the wording of the popover, AFAIK, we haven't done steps in a popover, ever. So I'd make it a simple sentence:
Go to the <<deployments page>>, select the remote deployment and copy the Elasticsearch endpoint URL.

yuliacech · 2021-04-01T15:14:11Z

@elasticmachine merge upstream

yuliacech · 2021-04-06T14:43:51Z

@elasticmachine merge upstream

mdefazio

Looks good. Sorry for any delay on checking the changes.

yuliacech · 2021-04-06T16:22:18Z

@elasticmachine merge upstream

yuliacech · 2021-04-06T17:40:02Z

@elasticmachine merge upstream

yuliacech · 2021-04-07T08:29:17Z

@elasticmachine merge upstream

…tic#94450) * Implemented in-form Cloud deployment url input * Fixed i18n files and added mode switch back for non-Cloud * Added cloud docs link to the documentation service, fixed snapshot tests * Fixed docs build * Added jest test for the new cloud url input * Added unit test for cloud validation * Fixed eslint error * Fixed ts errors * Added ts-ignore * Fixed ts errors * Refactored connection mode component and component tests * Fixed import * Fixed copy * Fixed copy * Reverted docs changes * Reverted docs changes * Replaced the screenshot with a popover and refactored integration tests * Added todo for cloud deployments link * Changed cloud URL help text copy * Added cloud base url and deleted unnecessary base path * Fixed es error * Fixed es error * Changed wording * Reverted docs changes * Updated the help popover * Deleted unneeded fragment component * Deleted unneeded fragment component * Updated tests descriptions to be more detailed Co-authored-by: Kibana Machine <[email protected]>

…) (#96558) * Implemented in-form Cloud deployment url input * Fixed i18n files and added mode switch back for non-Cloud * Added cloud docs link to the documentation service, fixed snapshot tests * Fixed docs build * Added jest test for the new cloud url input * Added unit test for cloud validation * Fixed eslint error * Fixed ts errors * Added ts-ignore * Fixed ts errors * Refactored connection mode component and component tests * Fixed import * Fixed copy * Fixed copy * Reverted docs changes * Reverted docs changes * Replaced the screenshot with a popover and refactored integration tests * Added todo for cloud deployments link * Changed cloud URL help text copy * Added cloud base url and deleted unnecessary base path * Fixed es error * Fixed es error * Changed wording * Reverted docs changes * Updated the help popover * Deleted unneeded fragment component * Deleted unneeded fragment component * Updated tests descriptions to be more detailed Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Kibana Machine <[email protected]>

kibanamachine · 2021-04-12T17:26:38Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request
Commit: e0d85ee
Storybooks Preview
Pipeline Steps (look for red circles / failed steps)
Interpreting CI Failures

Failed CI Steps

Test Failures

Kibana Pipeline / general / X-Pack EPM API Integration Tests.x-pack/test/fleet_api_integration/apis/epm/list·ts.Fleet Endpoints EPM Endpoints EPM - list "before all" hook in "EPM - list"

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 1 times on tracked branches: https://dryrun

[00:00:00]       │
[00:00:00]         └-: Fleet Endpoints
[00:00:00]           └-> "before all" hook in "Fleet Endpoints"
[00:00:00]           └-: EPM Endpoints
[00:00:00]             └-> "before all" hook in "EPM Endpoints"
[00:00:14]             └-: EPM - list
[00:00:14]               └-> "before all" hook in "EPM - list"
[00:00:14]               └-> "before all" hook in "EPM - list"
[00:00:14]                 │ info [fleet/empty_fleet_server] Loading "mappings.json"
[00:00:14]                 └- ✖ fail: Fleet Endpoints EPM Endpoints EPM - list "before all" hook in "EPM - list"
[00:00:14]                 │      ResponseError: resource_already_exists_exception
[00:00:14]                 │       at onBody (/dev/shm/workspace/parallel/2/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:337:23)
[00:00:14]                 │       at IncomingMessage.onEnd (/dev/shm/workspace/parallel/2/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:264:11)
[00:00:14]                 │       at endReadableNT (internal/streams/readable.js:1327:12)
[00:00:14]                 │       at processTicksAndRejections (internal/process/task_queues.js:80:21)
[00:00:14]                 │ 
[00:00:14]                 │

Stack Trace

ResponseError: resource_already_exists_exception
    at onBody (/dev/shm/workspace/parallel/2/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:337:23)
    at IncomingMessage.onEnd (/dev/shm/workspace/parallel/2/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:264:11)
    at endReadableNT (internal/streams/readable.js:1327:12)
    at processTicksAndRejections (internal/process/task_queues.js:80:21) {
  meta: {
    body: { error: [Object], status: 400 },
    statusCode: 400,
    headers: {
      'content-type': 'application/json;charset=utf-8',
      'content-length': '405'
    },
    meta: {
      context: null,
      request: [Object],
      name: 'elasticsearch-js',
      connection: [Object],
      attempts: 0,
      aborted: false
    }
  }
}

Kibana Pipeline / general / X-Pack EPM API Integration Tests.x-pack/test/fleet_api_integration/apis/epm/list·ts.Fleet Endpoints EPM Endpoints EPM - list "after all" hook in "EPM - list"

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 1 times on tracked branches: https://dryrun

[00:00:00]       │
[00:00:00]         └-: Fleet Endpoints
[00:00:00]           └-> "before all" hook in "Fleet Endpoints"
[00:00:00]           └-: EPM Endpoints
[00:00:00]             └-> "before all" hook in "EPM Endpoints"
[00:00:14]             └-: EPM - list
[00:00:14]               └-> "before all" hook in "EPM - list"
[00:00:14]               └-> "before all" hook in "EPM - list"
[00:00:14]                 │ info [fleet/empty_fleet_server] Loading "mappings.json"
[00:00:14]                 └- ✖ fail: Fleet Endpoints EPM Endpoints EPM - list "before all" hook in "EPM - list"
[00:00:14]                 │      ResponseError: resource_already_exists_exception
[00:00:14]                 │       at onBody (/dev/shm/workspace/parallel/2/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:337:23)
[00:00:14]                 │       at IncomingMessage.onEnd (/dev/shm/workspace/parallel/2/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:264:11)
[00:00:14]                 │       at endReadableNT (internal/streams/readable.js:1327:12)
[00:00:14]                 │       at processTicksAndRejections (internal/process/task_queues.js:80:21)
[00:00:14]                 │ 
[00:00:14]                 │ 
[00:00:14]                 └-> "after all" hook in "EPM - list"
[00:00:14]                   │ info [fleet/empty_fleet_server] Loading "mappings.json"
[00:00:14]                   └- ✖ fail: Fleet Endpoints EPM Endpoints EPM - list "after all" hook in "EPM - list"
[00:00:14]                   │      ResponseError: resource_already_exists_exception
[00:00:14]                   │       at onBody (/dev/shm/workspace/parallel/2/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:337:23)
[00:00:14]                   │       at IncomingMessage.onEnd (/dev/shm/workspace/parallel/2/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:264:11)
[00:00:14]                   │       at endReadableNT (internal/streams/readable.js:1327:12)
[00:00:14]                   │       at processTicksAndRejections (internal/process/task_queues.js:80:21)
[00:00:14]                   │ 
[00:00:14]                   │

Stack Trace

ResponseError: resource_already_exists_exception
    at onBody (/dev/shm/workspace/parallel/2/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:337:23)
    at IncomingMessage.onEnd (/dev/shm/workspace/parallel/2/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:264:11)
    at endReadableNT (internal/streams/readable.js:1327:12)
    at processTicksAndRejections (internal/process/task_queues.js:80:21) {
  meta: {
    body: { error: [Object], status: 400 },
    statusCode: 400,
    headers: {
      'content-type': 'application/json;charset=utf-8',
      'content-length': '405'
    },
    meta: {
      context: null,
      request: [Object],
      name: 'elasticsearch-js',
      connection: [Object],
      attempts: 0,
      aborted: false
    }
  }
}

Kibana Pipeline / general / X-Pack EPM API Integration Tests.x-pack/test/fleet_api_integration/apis/epm/list·ts.Fleet Endpoints EPM Endpoints EPM - list "before all" hook in "EPM - list"

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]       │
[00:00:00]         └-: Fleet Endpoints
[00:00:00]           └-> "before all" hook in "Fleet Endpoints"
[00:00:00]           └-: EPM Endpoints
[00:00:00]             └-> "before all" hook in "EPM Endpoints"
[00:00:14]             └-: EPM - list
[00:00:14]               └-> "before all" hook in "EPM - list"
[00:00:14]               └-> "before all" hook in "EPM - list"
[00:00:14]                 │ info [fleet/empty_fleet_server] Loading "mappings.json"
[00:00:14]                 └- ✖ fail: Fleet Endpoints EPM Endpoints EPM - list "before all" hook in "EPM - list"
[00:00:14]                 │      ResponseError: resource_already_exists_exception
[00:00:14]                 │       at onBody (/dev/shm/workspace/parallel/2/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:337:23)
[00:00:14]                 │       at IncomingMessage.onEnd (/dev/shm/workspace/parallel/2/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:264:11)
[00:00:14]                 │       at endReadableNT (internal/streams/readable.js:1327:12)
[00:00:14]                 │       at processTicksAndRejections (internal/process/task_queues.js:80:21)
[00:00:14]                 │ 
[00:00:14]                 │

Stack Trace

ResponseError: resource_already_exists_exception
    at onBody (/dev/shm/workspace/parallel/2/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:337:23)
    at IncomingMessage.onEnd (/dev/shm/workspace/parallel/2/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:264:11)
    at endReadableNT (internal/streams/readable.js:1327:12)
    at processTicksAndRejections (internal/process/task_queues.js:80:21) {
  meta: {
    body: { error: [Object], status: 400 },
    statusCode: 400,
    headers: {
      'content-type': 'application/json;charset=utf-8',
      'content-length': '405'
    },
    meta: {
      context: null,
      request: [Object],
      name: 'elasticsearch-js',
      connection: [Object],
      attempts: 0,
      aborted: false
    }
  }
}

and 2 more failures, only showing the first 3.

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`remoteClusters`	131	138	+7

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`remoteClusters`	160.0KB	165.6KB	+5.6KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`remoteClusters`	14.1KB	14.2KB	+143.0B

History

💚 Build #118161 succeeded e0d85ee
💚 Build #118129 succeeded 9c46850
💔 Build #117830 failed 205aa4f
💚 Build #117767 succeeded a2c811e

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Implemented in-form Cloud deployment url input

e88f2d9

yuliacech added release_note:enhancement Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more v7.13.0 v8.0.0 labels Mar 11, 2021

Fixed i18n files and added mode switch back for non-Cloud

61eee0b

yuliacech marked this pull request as ready for review March 11, 2021 16:58

yuliacech requested a review from a team as a code owner March 11, 2021 16:58

yuliacech mentioned this pull request Mar 11, 2021

[Remote clusters] Cloud deployment form when adding new cluster #93953

Closed

yuliacech requested review from mdefazio and zanbel March 11, 2021 17:08

Merge branch 'master' into remote_clusters_cloud_form_2

134d063

kibanamachine and others added 2 commits March 16, 2021 09:51

Merge branch 'master' into remote_clusters_cloud_form_2

24ee218

Added cloud docs link to the documentation service, fixed snapshot tests

800c9ce

yuliacech requested a review from a team as a code owner March 16, 2021 14:47

afharo approved these changes Mar 16, 2021

View reviewed changes

kibanamachine and others added 3 commits March 17, 2021 04:04

Merge branch 'master' into remote_clusters_cloud_form_2

2631c2c

Fixed docs build

28d6fb0

Added jest test for the new cloud url input

907ae2f

Merge branch 'master' into remote_clusters_cloud_form_2

94d626b

Merge branch 'master' into remote_clusters_cloud_form_2

d7e1417

yuliacech and others added 2 commits March 18, 2021 16:22

Merge branch 'master' into remote_clusters_cloud_form_2

82eae18

Added unit test for cloud validation

1ac351a

yuliacech added 4 commits March 31, 2021 16:34

Fixed es error

bfa86b7

Fixed es error

ee1af7c

Changed wording

8dafc20

Reverted docs changes

f5c1dba

kibanamachine and others added 2 commits April 1, 2021 11:14

Merge branch 'master' into remote_clusters_cloud_form_2

85655c1

Updated the help popover

d157ddd

Merge branch 'master' into remote_clusters_cloud_form_2

2fa7a72

mdefazio approved these changes Apr 6, 2021

View reviewed changes

Merge branch 'master' into remote_clusters_cloud_form_2

95a06eb

Merge branch 'master' into remote_clusters_cloud_form_2

9e2ab0d

kibanamachine and others added 4 commits April 7, 2021 04:29

Merge branch 'master' into remote_clusters_cloud_form_2

a2c811e

Deleted unneeded fragment component

205aa4f

Deleted unneeded fragment component

9c46850

Updated tests descriptions to be more detailed

e0d85ee

yuliacech merged commit 0316787 into elastic:master Apr 8, 2021

yuliacech mentioned this pull request Apr 8, 2021

[7.x] [Remote clusters] Cloud deployment form when adding new cluster (#94450) #96558

Merged

This was referenced Apr 21, 2021

Optimize Remote Clusters UX on Cloud #90894

Closed

[Remote Clusters] Update details flyout on Cloud #97831

Open

yuliacech deleted the remote_clusters_cloud_form_2 branch April 22, 2021 15:38

cjcenizal added the Feature:CCR and Remote Clusters label Apr 30, 2021

cjcenizal added the NeededFor:Cloud label Mar 7, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Remote clusters] Cloud deployment form when adding new cluster #94450

[Remote clusters] Cloud deployment form when adding new cluster #94450

yuliacech commented Mar 11, 2021 •

edited

Loading

elasticmachine commented Mar 11, 2021

zanbel commented Mar 15, 2021

yuliacech commented Mar 16, 2021

afharo left a comment

yuliacech commented Mar 17, 2021

mdefazio commented Mar 18, 2021

yuliacech commented Mar 18, 2021

yuliacech commented Mar 18, 2021

kellyemurphy commented Mar 31, 2021

yuliacech commented Apr 1, 2021

yuliacech commented Apr 6, 2021

mdefazio left a comment

yuliacech commented Apr 6, 2021

yuliacech commented Apr 6, 2021

yuliacech commented Apr 7, 2021

kibanamachine commented Apr 12, 2021 •

edited

Loading

Standard Out

Stack Trace

Standard Out

Stack Trace

Standard Out

Stack Trace

[Remote clusters] Cloud deployment form when adding new cluster #94450

[Remote clusters] Cloud deployment form when adding new cluster #94450

Conversation

yuliacech commented Mar 11, 2021 • edited Loading

Summary

Screenshot

Implementation

Remote clusters on Cloud context

Cloud endpoint url input

Screenshot and GIF

How to test

Checklist

Release Note

elasticmachine commented Mar 11, 2021

zanbel commented Mar 15, 2021

yuliacech commented Mar 16, 2021

afharo left a comment

Choose a reason for hiding this comment

yuliacech commented Mar 17, 2021

mdefazio commented Mar 18, 2021

yuliacech commented Mar 18, 2021

yuliacech commented Mar 18, 2021

kellyemurphy commented Mar 31, 2021

yuliacech commented Apr 1, 2021

yuliacech commented Apr 6, 2021

mdefazio left a comment

Choose a reason for hiding this comment

yuliacech commented Apr 6, 2021

yuliacech commented Apr 6, 2021

yuliacech commented Apr 7, 2021

kibanamachine commented Apr 12, 2021 • edited Loading

💔 Build Failed

Failed CI Steps

Test Failures

Standard Out

Stack Trace

Standard Out

Stack Trace

Standard Out

Stack Trace

Metrics [docs]

Module Count

Async chunks

Page load bundle

History

yuliacech commented Mar 11, 2021 •

edited

Loading

kibanamachine commented Apr 12, 2021 •

edited

Loading