Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Console] Proxy fallback #50185

Merged
merged 10 commits into from
Nov 27, 2019
Merged

[Console] Proxy fallback #50185

merged 10 commits into from
Nov 27, 2019

Conversation

jloleysens
Copy link
Contributor

@jloleysens jloleysens commented Nov 11, 2019 •
edited
Loading

Summary

Attempt at addressing this issue: #32845

Need review on the idea for further discussion - this is loosely based on the round robin strategy for switching between nodes in a cluster inside of the elasticsearch js client. We could use the client directly but this may make preserving the current proxy behaviour more difficult.

How to review

  • Read through the strategy for getting the next available host
  • Are there any edge cases this opens that we don't want?
  • On the UI, does the user need to know which node we are currently talking to? (i.e., would this be relevant to show inside of Console?)

How to test

In order to test proxy fallback you need to have ES running in Clustered mode. For this you need at least 3 nodes, each configured as eligible for being a master node with an initial discovery node configured (more information here: https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html). Once that is done:

  1. Configure Kibana:
    1. Set hosts kibana.dev.yml to point to all the nodes in your 3 node cluster
    2. I found that I had uncomment that username and password settings in kibana.yml too.
  2. Start Kibana, go to Console and run a test command.
  3. Kill the ES process Console was connecting to to service requests. In the current implementation this will be the first configured hosts: [] entry in kibana.yml.

@jloleysens jloleysens added discuss Feature:Console Dev Tools Console Feature release_note:fix v8.0.0 Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more v7.6.0 labels Nov 11, 2019
@elasticmachine
Copy link
Contributor

Pinging @elastic/es-ui (Team:Elasticsearch UI)

@jloleysens jloleysens changed the title [Console] Console proxy fallback [Console] Proxy fallback Nov 11, 2019
@elasticmachine
Copy link
Contributor

💔 Build Failed

@jloleysens
Copy link
Contributor Author

@elasticmachine merge upstream

@elasticmachine
Copy link
Contributor

💔 Build Failed

@jloleysens
Copy link
Contributor Author

@elasticmachine merge upstream

@jloleysens
Merge branch 'master' of github.com:elastic/kibana into console-proxy…
31fdcec 
…-fallback

* 'master' of github.com:elastic/kibana:
  Remove internal platform types exports (elastic#50427)
  [APM] Document `apm_oss.metricsIndices` and `apm_oss.sourcemap… (elastic#50312)
  [Telemetry] Server side fetcher (elastic#50015)
  [SIEM] Detection engine placeholders (elastic#50220)
  [Uptime] Donut chart loader position centered vertically  (elastic#50219)
  update telemetry banner notice text (elastic#50403)
  Fix aborting when searching without batching (elastic#49966)
  [Telemetry] Remove telemetry splash page and add conditional messaging (elastic#50189)
  Revert chromedriver update (elastic#50324)
  Remove deprecated argument include_type_name from ES calls (elastic#50285)
  [Maps] add settings to maps telemetry (elastic#50161)
  remove visualize loader (elastic#46910)
  Fix misuse of react-router and react-router-dom (elastic#50120)
  Move table-list-view to kibana-react (elastic#50046)
  [ML] Stats bar for data frame analytics (elastic#49464)
@elasticmachine
Copy link
Contributor

💔 Build Failed

@jloleysens jloleysens marked this pull request as ready for review November 18, 2019 21:31
@jloleysens
Merge branch 'master' of github.com:elastic/kibana into console-proxy…
bbee1ef 
…-fallback

* 'master' of github.com:elastic/kibana: (116 commits)
  [Maps] move apply global filter settting from layer to source (elastic#50523)
  [SIEM] Fix: Empty `Source` / `Destination` shown when only ports are populated (elastic#50843)
  [Maps] Delay vector tile layer syncing until spritesheet is loaded (elastic#48955)
  [Maps] prevent users from overflowing URL when filtering by shape (elastic#50747)
  [DOCS] Mark Beats central management as discontinued (elastic#49423)
  [page_objects/common_page] convert to ts (elastic#50771)
  [NP Kibana Migrations ] kibana plugin home (elastic#50444)
  [DOCS] Shareables naming convention (elastic#50497)
  [ML] DF Analytics - auto-populate model_memory_limit (elastic#50714)
  Increase alerting test stability and reduce flakiness (elastic#50246)
  [ML] Remaning new_job_new folder (elastic#50917)
  [Telemetry] Show opt-in changes for OSS users (elastic#50831)
  [ML] Fix lat_long anomalies table links menu and value formatting (elastic#50916)
  [Dev] Fix serialising a really big string (elastic#50915)
  Better explanation about the Prettier recommendation (extension vs. NPM module) (elastic#50629)
  [Monitoring] Use a basic monitoring user for tests (elastic#47865)
  [Monitoring] Gracefully handle issue with filebeat indices (elastic#48929)
  [Monitoring] Improve permissions required around setup mode (elastic#50421)
  Additional validation for elasticsearch username (elastic#48247)
  Revert changes to use_kibana_ui_setting (elastic#50877)
  ...

# Conflicts:
#	src/legacy/core_plugins/console/server/request.test.ts
@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

alisonelizabeth
alisonelizabeth approved these changes Nov 26, 2019
View reviewed changes
Copy link
Contributor

@alisonelizabeth alisonelizabeth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jloleysens Code LGTM. I tested locally and confirmed the fix.

On the UI, does the user need to know which node we are currently talking to? (i.e., would this be relevant to show inside of Console?)

This is a good question. I think it could be helpful, and I would be interested in hearing feedback from others.

@jbudz
Copy link
Member

jbudz commented Nov 26, 2019

+1, the user can always check with an api call to see node info. Useful but don't think it's required for this PR

@jloleysens
Copy link
Contributor Author

@elasticmachine merge upstream

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@jloleysens jloleysens merged commit 7c31512 into elastic:master Nov 27, 2019
@jloleysens jloleysens deleted the console-proxy-fallback branch November 27, 2019 11:33
jloleysens added a commit to jloleysens/kibana that referenced this pull request Nov 27, 2019
@jloleysens
[Console] Proxy fallback (elastic#50185)
4cd2805 
* First iteration of liveness manager for Console

* First iteration of PoC working

* Updated console proxy fallback behaviour after feedback

* remove @types/node-fetch

* If all hosts failed due to connection refused errors 502

* Remove unnecessary existence check
@jloleysens jloleysens mentioned this pull request Nov 27, 2019
Merged
jloleysens added a commit that referenced this pull request Nov 27, 2019
@jloleysens
[Console] Proxy fallback (#50185) (#51814)
f62d59a 
* First iteration of liveness manager for Console

* First iteration of PoC working

* Updated console proxy fallback behaviour after feedback

* remove @types/node-fetch

* If all hosts failed due to connection refused errors 502

* Remove unnecessary existence check
jloleysens added a commit to jloleysens/kibana that referenced this pull request Nov 27, 2019
@jloleysens
Merge remote-tracking branch 'upstream/7.x' into backport/7.x/pr-50163
bc6197f 
* upstream/7.x:
  Fix infinite redirect loop when multiple cookies are sent (elastic#50452) (elastic#51821)
  [Console] Proxy fallback (elastic#50185) (elastic#51814)
  Added endgame-* index and new heading 3 Elastic Endpoint SMP. (elastic#51071) (elastic#51828)
  [Maps] Added options to disable zoom, hide tool tips, widgets/overlays in embeddable maps (elastic#50663) (elastic#51811)
  Move errors and validate index pattern ⇒ NP (elastic#51805) (elastic#51831)
  [SIEM][Detection Engine] Adds ecs threat properties to rules (elastic#51782) (elastic#51827)
  [Lens] Remove client-side reference to server source code (elastic#51763) (elastic#51825)
  fixes drag and drop in tests (elastic#51806) (elastic#51813)
  [Uptime] Redesign/44541  new monitor list expanded row (elastic#46567) (elastic#51809)
  [7.x] [Telemetry] collector set to np (elastic#51618) (elastic#51787)
  [Uptime] added test for chart wrapper (elastic#50399) (elastic#51808)
  Expressions service fixes: better error and loading states handling (elastic#51183) (elastic#51800)
  Query String(Bar) Input - cleanup (elastic#51598) (elastic#51804)
  [ML] Adjust and re-enable categorization advanced wizard test (elastic#51005) (elastic#51017)
  fixes url state tests (elastic#51746) (elastic#51798)
  fixes browser field tests (elastic#51738) (elastic#51799)
  [Task Manager] Tests for the ability to run tasks of varying durations in parallel (elastic#51572) (elastic#51701)
  [ML] Fix anomaly detection test suite (elastic#51712) (elastic#51795)
  [SIEM] Fix Timeline drag and drop behavior (elastic#51558) (elastic#51793)
@jloleysens jloleysens mentioned this pull request Nov 28, 2019
Closed
mbondyra added a commit to mbondyra/kibana that referenced this pull request Nov 28, 2019
@mbondyra
Merge branch 'IS-46410_remove-@kbn/ui-framework' of github.com:mbondy…
4b0ac16 
…ra/kibana into IS-46410_remove-@kbn/ui-framework

* 'IS-46410_remove-@kbn/ui-framework' of github.com:mbondyra/kibana: (49 commits)
  [ML] Re-activate after method in transform test (elastic#51815)
  [SIEM] [Detection Engine] Add edit on rule creation (elastic#51670)
  De-angularize visLegend (elastic#50613)
  [SIEM][Detection Engine] Change security model to use SIEM permissions
  [Monitoring] Sass cleanup (elastic#51100)
  Move errors and validate index pattern ⇒ NP (elastic#51805)
  fixes pagination tests (elastic#51822)
  Split legacy plugin discovery, expose SavedObjects scopedClient, wrappers, repository (elastic#48882)
  [SIEM][Detection Engine] Adds ecs threat properties to rules (elastic#51782)
  [Lens] Remove client-side reference to server source code (elastic#51763)
  Fix infinite redirect loop when multiple cookies are sent (elastic#50452)
  fixes drag and drop in tests (elastic#51806)
  [Console] Proxy fallback (elastic#50185)
  Query String(Bar) Input - cleanup (elastic#51598)
  shim visualizations plugin (elastic#50624)
  Expressions service fixes: better error and loading states handling (elastic#51183)
  fixes url state tests (elastic#51746)
  fixes browser field tests (elastic#51738)
  [ML] Fix anomaly detection test suite (elastic#51712)
  [SIEM] Fix Timeline drag and drop behavior (elastic#51558)
  ...
jloleysens added a commit to jloleysens/kibana that referenced this pull request Nov 28, 2019
@jloleysens
Merge branch 'master' of github.com:elastic/kibana into np-migration/…
65e74cf 
…license-management

* 'master' of github.com:elastic/kibana: (48 commits)
  Enable alerting and actions plugin by default (elastic#51254)
  Fix error returned when creating an alert with ES security disabled (elastic#51639)
  [Discover] Improve Percy functional tests (elastic#51699)
  fixes timeline data providers tests (elastic#51862)
  [Dependencies]: upgrade react to latest v16.12.0 (elastic#51145)
  Allow routes to define some payload config values (elastic#50783)
  Move saved queries service + language switcher ⇒ NP (elastic#51812)
  [ML] Re-activate after method in transform test (elastic#51815)
  [SIEM] [Detection Engine] Add edit on rule creation (elastic#51670)
  De-angularize visLegend (elastic#50613)
  [SIEM][Detection Engine] Change security model to use SIEM permissions
  [Monitoring] Sass cleanup (elastic#51100)
  Move errors and validate index pattern ⇒ NP (elastic#51805)
  fixes pagination tests (elastic#51822)
  Split legacy plugin discovery, expose SavedObjects scopedClient, wrappers, repository (elastic#48882)
  [SIEM][Detection Engine] Adds ecs threat properties to rules (elastic#51782)
  [Lens] Remove client-side reference to server source code (elastic#51763)
  Fix infinite redirect loop when multiple cookies are sent (elastic#50452)
  fixes drag and drop in tests (elastic#51806)
  [Console] Proxy fallback (elastic#50185)
  ...
@jloleysens jloleysens mentioned this pull request Apr 17, 2020
Merged
1 task
@kiju98
Copy link

kiju98 commented Sep 2, 2020 •
edited
Loading

https://www.elastic.co/guide/en/kibana/7.6/production.html#high-availability still mentions Currently the Console application is limited to connecting to the first node listed. Is it a doc being not update or this merge didn't make it into 7.6?

@jloleysens
Copy link
Contributor Author

Hi @kiju98 ! I was not aware of the docs at the time, it should probably have been updated :).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alisonelizabeth alisonelizabeth alisonelizabeth approved these changes

@jbudz jbudz jbudz approved these changes

Assignees
No one assigned
Labels
discuss Feature:Console Dev Tools Console Feature release_note:fix Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more v7.6.0 v8.0.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@jloleysens @elasticmachine @jbudz @kiju98 @alisonelizabeth