Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[data.search] Handle warnings inside of headers #103744

Merged
merged 16 commits into from
Aug 17, 2021

Conversation

lukasolson
Copy link
Member

@lukasolson lukasolson commented Jun 29, 2021

Summary

Closes #98764.

This PR adds support to the data.search services to surface warning properties that are returned in headers from Elasticsearch responses. It also adds developer examples and adds support to SearchSource for handling these types of warnings directly.

Checklist

Delete any items that are not applicable to this PR.

Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to identify risks that should be tested prior to the change/feature release.

When forming the risk matrix, consider some of the following examples and how they may potentially impact the change:

Risk Probability Severity Mitigation/Notes
Multiple Spaces—unexpected behavior in non-default Kibana Space. Low High Integration tests will verify that all features are still supported in non-default Kibana Space and when user switches between spaces.
Multiple nodes—Elasticsearch polling might have race conditions when multiple Kibana nodes are polling for the same tasks. High Low Tasks are idempotent, so executing them multiple times will not result in logical error, but will degrade performance. To test for this case we add plenty of unit tests around this logic and document manual testing procedure.
Code should gracefully handle cases when feature X or plugin Y are disabled. Medium High Unit tests will verify that any feature flag or plugin combination still results in our service operational.
See more potential risk examples

For maintainers

Plugin API changes

The data.search service now returns a warning property that includes any warnings returned from Elasticsearch in the headers.

@lukasolson lukasolson requested a review from a team as a code owner June 29, 2021 19:19
@lukasolson lukasolson self-assigned this Jun 29, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-app-services (Team:AppServices)

@lukasolson lukasolson added Feature:Search Querying infrastructure in Kibana v7.15.0 v8.0.0 labels Jun 29, 2021
@lukasolson lukasolson added the release_note:plugin_api_changes Contains a Plugin API changes section for the breaking plugin API changes section. label Jul 1, 2021
@lukasolson lukasolson requested review from lizozom and Dosant July 7, 2021 14:04
@lukasolson
Copy link
Member Author

@elasticmachine merge upstream

@spalger
Copy link
Contributor

spalger commented Jul 8, 2021

jenkins, test this

(had to abort for Jenkins upgrade)

@lizozom
Copy link
Contributor

lizozom commented Jul 13, 2021

Following up on our call - I think it would be better if the search API exposed two fields:

  • status
  • message

In this PR we could use it only for warnings, but I would make a follow up PR to get rid of isPartial and isRunning as well..

@lukasolson
Copy link
Member Author

@elasticmachine merge upstream

@lukasolson
Copy link
Member Author

Following up on our call - I think it would be better if the search API exposed two fields:

  • status
  • message

I was giving this some more thought today and I'm not sure this will work, as a search can complete and still have warnings. So I'm not sure what the enum of statuses would look like. Maybe we can chat a little bit more about this approach or possibly move it to a follow-up PR.

@lizozom
Copy link
Contributor

lizozom commented Jul 15, 2021

@lukasolson

Then what about aiming for -

// Response format
{
   body: ...,
   status: OK \NOT_OK
   messages: [{
     test: 'This is a warning',
     level: ' warning',
   }, {
     test: 'This is an error',
     level: ' error',
   }]
}

A successful response will have an OK status, but may or may not have a warning(s?).
A failed response will have a NOT_OK status with an one or more errors or warnings.

@lukasolson
Copy link
Member Author

@lizozom I removed the isWarningResponse helper, as it isn't super helpful (since it just checked if the warning property exists). I would like to chat more about your proposed changes to have messages, but ultimately I think it would make sense to move it to a separate PR so we can make sure to handle errors properly.

@lukasolson
Copy link
Member Author

@elasticmachine merge upstream

@lukasolson lukasolson requested a review from lizozom August 11, 2021 19:42
@lukasolson
Copy link
Member Author

@elasticmachine merge upstream

@lizozom
Copy link
Contributor

lizozom commented Aug 16, 2021

@elasticmachine merge upstream

Copy link
Contributor

@lizozom lizozom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, lets merge as is and wait for feedback 👍🏻

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
data 791.0KB 791.2KB +232.0B
Unknown metric groups

API count

id before after diff
data 3480 3482 +2

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @lukasolson

@Dosant
Copy link
Contributor

Dosant commented Aug 16, 2021

It is interesting that one of our sample dashboards shows a real warning now:

Screen Shot 2021-08-16 at 13 17 22

Seems like the warning is a deprecation message which shouldn't concern our users, but should be addressed by us instead.
With this as an example, looks like we can't move forward with the change as is? Because this might produce false warnings for real users? 😞

Seems like we need a way to separate real or actionable by-user warnings from such deprecation warnings which should be handled by us? Maybe there are other types of warnings that we have to mute :(

@lukasolson
Copy link
Member Author

lukasolson commented Aug 17, 2021

@Dosant great find! IMHO, we want to expose these errors in a noticeable way in the UX so that this exact sort of issue doesn't happen in the future. I've opened an issue to track this: #108813

@lukasolson
Copy link
Member Author

Also opened a fix: #108825

@lukasolson lukasolson merged commit 6a39dc1 into elastic:master Aug 17, 2021
@Dosant
Copy link
Contributor

Dosant commented Aug 17, 2021

@Dosant great find! IMHO, we want to expose these errors in a noticeable way in the UX so that this exact sort of issue doesn't happen in the future. I've opened an issue to track this: #108813

Awesome!

@kibanamachine
Copy link
Contributor

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create backports run node scripts/backport --pr 103744 or prevent reminders by adding the backport:skip label.

@kibanamachine kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Aug 19, 2021
lukasolson added a commit to lukasolson/kibana that referenced this pull request Aug 19, 2021
* [data.search] Handle warnings inside of headers

* Update docs

* Add tests

* Remove isWarningResponse

Co-authored-by: Kibana Machine <[email protected]>
lukasolson added a commit to lukasolson/kibana that referenced this pull request Aug 19, 2021
* [data.search] Handle warnings inside of headers

* Update docs

* Add tests

* Remove isWarningResponse

Co-authored-by: Kibana Machine <[email protected]>
@kibanamachine
Copy link
Contributor

Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync.

lukasolson added a commit that referenced this pull request Aug 20, 2021
* [data.search] Handle warnings inside of headers

* Update docs

* Add tests

* Remove isWarningResponse

Co-authored-by: Kibana Machine <[email protected]>

Co-authored-by: Kibana Machine <[email protected]>
@kibanamachine kibanamachine removed the backport missing Added to PRs automatically when the are determined to be missing a backport. label Aug 20, 2021
lukasolson added a commit that referenced this pull request Aug 20, 2021
* [data.search] Handle warnings inside of headers

* Update docs

* Add tests

* Remove isWarningResponse

Co-authored-by: Kibana Machine <[email protected]>

Co-authored-by: Kibana Machine <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Search Querying infrastructure in Kibana release_note:plugin_api_changes Contains a Plugin API changes section for the breaking plugin API changes section. v7.15.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[data.search] Cross-cluster search error handling
6 participants