[Monitoring] Gracefully handle issue with filebeat indices #48929

chrisronline · 2019-10-22T16:07:22Z

Resolves #48533

Feel free to suggest better copy!

To test,

Install and configure filebeat to ship elasticsearch logs to your local cluster
Enable monitoring
Open stack monitoring UI and verify the logs are visible
In dev tools, run DELETE filebeat-*
Run some more commands in dev tools to generate more ES logs
Check GET _cat/indices and verify you see an filebeat-${VERSION} index
Go back and refresh the stack monitoring UI and verify the page loads with the logs panel showing the error shown above

elasticmachine · 2019-10-22T16:07:24Z

Pinging @elastic/stack-monitoring (Team:Monitoring)

cachedout · 2019-10-22T16:19:52Z

I will test this shortly and do a code review but in general, is the precedence for linking to GH issues from the interface? It seems like the remediation steps are straightforward enough that we could probably just include them in the copy we display in the app.

chrisronline · 2019-10-22T16:33:44Z

@cachedout Good question. I'm not sure - I wasn't sure we had enough space to contain the full steps to fix this.

elasticmachine · 2019-10-22T17:26:54Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request
Commit: fb2dd25

cachedout · 2019-10-23T09:32:22Z

I played around with options and this is the shortest copy I could come up with that fit in the constraints of the box on the page:

There is an issue reading from your filebeat indices. Stop Filebeat, delete indices that start with `filebeat-*` and then restart Filebeat.```

That said, I don't have any objection to just linking to the issue, and that may actually be a better experience for the user.

cachedout

I left an alternate for the copy in the comments of this PR if you want to use it but I think it's just fine to merge as-is.

chrisronline · 2019-10-23T13:46:04Z

There is an issue reading from your filebeat indices. Stop Filebeat, delete indices that start with filebeat-* and then restart Filebeat.

My only concern with this is users don't technically need to delete the ILM-based indices, such as filebeat-8.0.0-2019.10.23-000001. Those will correctly match the template - it's only filebeat-${VERSION} indices that are affected by this

ycombinator · 2019-10-23T13:49:14Z

Just to offer an alternate solution: if we're going to link off to somewhere, perhaps we should link off to a section in https://www.elastic.co/guide/en/kibana/current/monitor-troubleshooting.html instead of a GitHub issue?

[EDIT] That said, I agree that if we can get away with not linking off to something externally but instead presenting the workaround in the app itself, that would be better.

cachedout · 2019-10-23T13:57:01Z

Just to offer an alternate solution: if we're going to link off to somewhere, perhaps we should link off to a section in elastic.co/guide/en/kibana/current/monitor-troubleshooting.html instead of a GitHub issue?

👍 to this suggestion. This is a great idea.

chrisronline · 2019-10-23T14:01:50Z

@cachedout Do you mind handling that PR separately? to update the that doc page to add this information? Then we can merge this PR with the updated link

Refs elastic#48929

igoristic

LGTM 👍

…ebeat_indices

elasticmachine · 2019-11-13T20:00:14Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request
Commit: 2a72ea7

cachedout · 2019-11-13T20:56:13Z

It's linked above but just to be clear, the PR with the copy for this is #49065

* Add filebeat notice Refs #48929 * Update docs/user/monitoring/monitoring-troubleshooting.asciidoc Co-Authored-By: Lisa Cawley <[email protected]> * Update docs/user/monitoring/monitoring-troubleshooting.asciidoc Co-Authored-By: Lisa Cawley <[email protected]> * Update docs/user/monitoring/monitoring-troubleshooting.asciidoc Co-Authored-By: Lisa Cawley <[email protected]> * Update docs/user/monitoring/monitoring-troubleshooting.asciidoc Co-Authored-By: Lisa Cawley <[email protected]> * Update docs/user/monitoring/monitoring-troubleshooting.asciidoc Co-Authored-By: Lisa Cawley <[email protected]> * Update docs/user/monitoring/monitoring-troubleshooting.asciidoc Co-Authored-By: Lisa Cawley <[email protected]> * Fixup per review comments

…ebeat_indices

chrisronline · 2019-11-14T17:19:01Z

@igoristic @cachedout I updated this PR with the trouble shooting link based on the changes @cachedout made in #49065. Ready for another round

elasticmachine · 2019-11-14T18:36:27Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 4c88e9b

x-pack/legacy/plugins/monitoring/server/lib/logs/detect_reason_from_exception.js

…ebeat_indices

elasticmachine · 2019-11-18T15:36:20Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: cf4e0e3

…8929) * Gracefully handle issue with filebeat indices * Update translation keys * Update link * Add in default, per PR comment

cachedout · 2019-11-18T15:58:27Z

I was mid-review when this got merged but I'm finished now and all looks well. :) 👍

chrisronline · 2019-11-18T15:59:04Z

@cachedout Ah, my bad! I figured it was good to go since it was a small change since your comment

…50923) * Gracefully handle issue with filebeat indices * Update translation keys * Update link * Add in default, per PR comment

…50924) * Gracefully handle issue with filebeat indices * Update translation keys * Update link * Add in default, per PR comment

chrisronline · 2019-11-18T20:36:50Z

Backport:

7.x: 4a336be
7.5: 3a476ec
7.4: c611890

…-fallback * 'master' of github.com:elastic/kibana: (116 commits) [Maps] move apply global filter settting from layer to source (elastic#50523) [SIEM] Fix: Empty `Source` / `Destination` shown when only ports are populated (elastic#50843) [Maps] Delay vector tile layer syncing until spritesheet is loaded (elastic#48955) [Maps] prevent users from overflowing URL when filtering by shape (elastic#50747) [DOCS] Mark Beats central management as discontinued (elastic#49423) [page_objects/common_page] convert to ts (elastic#50771) [NP Kibana Migrations ] kibana plugin home (elastic#50444) [DOCS] Shareables naming convention (elastic#50497) [ML] DF Analytics - auto-populate model_memory_limit (elastic#50714) Increase alerting test stability and reduce flakiness (elastic#50246) [ML] Remaning new_job_new folder (elastic#50917) [Telemetry] Show opt-in changes for OSS users (elastic#50831) [ML] Fix lat_long anomalies table links menu and value formatting (elastic#50916) [Dev] Fix serialising a really big string (elastic#50915) Better explanation about the Prettier recommendation (extension vs. NPM module) (elastic#50629) [Monitoring] Use a basic monitoring user for tests (elastic#47865) [Monitoring] Gracefully handle issue with filebeat indices (elastic#48929) [Monitoring] Improve permissions required around setup mode (elastic#50421) Additional validation for elasticsearch username (elastic#48247) Revert changes to use_kibana_ui_setting (elastic#50877) ... # Conflicts: # src/legacy/core_plugins/console/server/request.test.ts

Gracefully handle issue with filebeat indices

fb2dd25

chrisronline added review release_note:fix Team:Monitoring Stack Monitoring team v8.0.0 v7.5.0 Feature:Stack Monitoring v7.6.0 labels Oct 22, 2019

chrisronline requested review from cachedout and igoristic October 22, 2019 16:07

chrisronline self-assigned this Oct 22, 2019

cachedout approved these changes Oct 23, 2019

View reviewed changes

cachedout added a commit to cachedout/kibana that referenced this pull request Oct 23, 2019

Add filebeat notice

a35f5e3

Refs elastic#48929

cachedout mentioned this pull request Oct 23, 2019

Add filebeat notice #49065

Merged

igoristic approved these changes Oct 23, 2019

View reviewed changes

chrisronline added 2 commits November 13, 2019 13:28

Merge remote-tracking branch 'elastic/master' into monitoring/bad_fil…

511ee1d

…ebeat_indices

Update translation keys

2a72ea7

Merge remote-tracking branch 'elastic/master' into monitoring/bad_fil…

07d9b47

…ebeat_indices

Update link

4c88e9b

chrisronline requested review from igoristic and cachedout November 14, 2019 17:18

igoristic reviewed Nov 17, 2019

View reviewed changes

x-pack/legacy/plugins/monitoring/server/lib/logs/detect_reason_from_exception.js Outdated Show resolved Hide resolved

igoristic self-requested a review November 17, 2019 23:17

chrisronline added 2 commits November 18, 2019 09:14

Merge remote-tracking branch 'elastic/master' into monitoring/bad_fil…

45c6da9

…ebeat_indices

Add in default, per PR comment

cf4e0e3

igoristic approved these changes Nov 18, 2019

View reviewed changes

chrisronline merged commit f92573a into elastic:master Nov 18, 2019

chrisronline deleted the monitoring/bad_filebeat_indices branch November 18, 2019 15:49

chrisronline added the v7.4.3 label Nov 18, 2019

chrisronline mentioned this pull request Nov 18, 2019

[7.x] [Monitoring] Gracefully handle issue with filebeat indices (#48929) #50923

Merged

This was referenced Nov 18, 2019

[7.5] [Monitoring] Gracefully handle issue with filebeat indices (#48929) #50924

Merged

[7.4] [Monitoring] Gracefully handle issue with filebeat indices (#48929) #50926

Merged

chrisronline added a commit that referenced this pull request Nov 18, 2019

[Monitoring] Gracefully handle issue with filebeat indices (#48929) (#…

4a336be

…50923) * Gracefully handle issue with filebeat indices * Update translation keys * Update link * Add in default, per PR comment

chrisronline added a commit that referenced this pull request Nov 18, 2019

[Monitoring] Gracefully handle issue with filebeat indices (#48929) (#…

3a476ec

…50924) * Gracefully handle issue with filebeat indices * Update translation keys * Update link * Add in default, per PR comment

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Monitoring] Gracefully handle issue with filebeat indices #48929

[Monitoring] Gracefully handle issue with filebeat indices #48929

chrisronline commented Oct 22, 2019

elasticmachine commented Oct 22, 2019

cachedout commented Oct 22, 2019

chrisronline commented Oct 22, 2019

elasticmachine commented Oct 22, 2019

cachedout commented Oct 23, 2019

cachedout left a comment

chrisronline commented Oct 23, 2019

ycombinator commented Oct 23, 2019 •

edited

Loading

cachedout commented Oct 23, 2019

chrisronline commented Oct 23, 2019

igoristic left a comment

elasticmachine commented Nov 13, 2019

cachedout commented Nov 13, 2019

chrisronline commented Nov 14, 2019

elasticmachine commented Nov 14, 2019

elasticmachine commented Nov 18, 2019

cachedout commented Nov 18, 2019

chrisronline commented Nov 18, 2019

chrisronline commented Nov 18, 2019

[Monitoring] Gracefully handle issue with filebeat indices #48929

[Monitoring] Gracefully handle issue with filebeat indices #48929

Conversation

chrisronline commented Oct 22, 2019

elasticmachine commented Oct 22, 2019

cachedout commented Oct 22, 2019

chrisronline commented Oct 22, 2019

elasticmachine commented Oct 22, 2019

💔 Build Failed

cachedout commented Oct 23, 2019

cachedout left a comment

Choose a reason for hiding this comment

chrisronline commented Oct 23, 2019

ycombinator commented Oct 23, 2019 • edited Loading

cachedout commented Oct 23, 2019

chrisronline commented Oct 23, 2019

igoristic left a comment

Choose a reason for hiding this comment

elasticmachine commented Nov 13, 2019

💔 Build Failed

cachedout commented Nov 13, 2019

chrisronline commented Nov 14, 2019

elasticmachine commented Nov 14, 2019

💚 Build Succeeded

elasticmachine commented Nov 18, 2019

💚 Build Succeeded

cachedout commented Nov 18, 2019

chrisronline commented Nov 18, 2019

chrisronline commented Nov 18, 2019

ycombinator commented Oct 23, 2019 •

edited

Loading