Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Discover] fix histogram min interval #53979

Merged

Conversation

markov00
Copy link
Member

@markov00 markov00 commented Jan 4, 2020

Summary

This commit fix the missing bars on the histogram of Discover.
Discover is passing a minInterval parameter in milliseconds that is not
the the minimum interval in milliseconds between the buckets. Calendar intervals
are not fixed and this value change depending on time range and the time zone.
To avoid patching the elastic-chart and upgrading the dependency we applied a temporary patch
to the histogram code. For the feature version this patch will be removed
and a correct minInterval computation will be done directly in elastic-charts.

fix #52152

to test:
add some data that spans multiple years (possibly leap years, last one was 2016)

node scripts/makelogs.js -c 1m -d 2200/1
// or import the esArchived dataset
node scripts/es_archiver.js --es-url elastic:changeme@localhost:9275 --kibana-url http://elastic:changeme@localhost:5675 load long_window_logstash

create the index pattern for that index: logstash-* or long_window_logstash*
check the behaviour on histogram when the timerange spans several years (including leap years) with different interval configuration. The issue is visible when:

  • visualizing a timerange with monthly intervals that contains data in the February bucket
    timezone: all
    time range: Nov 1, 2017 @ 01:00:00.000 - Mar 21, 2018 @ 02:00:00.000
    intervals: Monthly

  • visualizing a timerange with a DST time change within the range. The DST change needs to be a positive change like when the time is moved forward (= that day last less than 24 hours) like in the following situation:
    timezone: Europe/Berlin (doesn't happen with UTC timezone)
    time range: Mar 1, 2018 @ 00:00:00.000 - May 1, 2018 @ 00:00:00.000
    intervals: all except Monthly and Yearly

  • visualizing a timerange with a DST time change with auto interval on a fixed/multi days interval:
    timezone: Europe/Berlin
    time range: Last 15 years - now
    intervals: every interval smaller than month (be sure that the scaled interval is scaled to 30days)

Checklist

Use strikethroughs to remove checklist items you don't feel are applicable to this PR.

For maintainers

@markov00 markov00 requested a review from a team January 4, 2020 13:35
@markov00 markov00 requested a review from a team as a code owner January 4, 2020 13:35
@markov00
Copy link
Member Author

markov00 commented Jan 4, 2020

@elasticmachine merge upstream

@markov00 markov00 added release_note:fix v7.6.0 v8.0.0 Team:Visualizations Visualization editors, elastic-charts and infrastructure labels Jan 5, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-app (Team:KibanaApp)

@markov00 markov00 added bug Fixes for quality problems that affect the customer experience Feature:Discover Discover Application labels Jan 5, 2020
@markov00
Copy link
Member Author

markov00 commented Jan 5, 2020

@elasticmachine merge upstream

@lizozom
Copy link
Contributor

lizozom commented Jan 6, 2020

@elasticmachine merge upstream

Copy link
Contributor

@lizozom lizozom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code changes LGTM

Test results:

I was able to reproduce all 3 errors on master and see them fixed on this branch.

(1) image

(2)
image

(3)
image

On master
(1)
image

(2)
image

(3)
image

@nickofthyme
Copy link
Contributor

@elasticmachine merge upstream

Copy link
Contributor

@nickofthyme nickofthyme left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, All scenarios are working as expected.

…s below.

- For some reason discover will error on getFieldsForWildcard without logstash_functional
- Loading long_window_logstash doesn't create an index pattern
- No other tests using long_window_logstash manually creates an index pattern
- No results found for long_window_logstash using abs time, until a relative time is selected in discover
@kibanamachine
Copy link
Contributor

💚 Build Succeeded

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@nickofthyme nickofthyme merged commit d5939c4 into elastic:master Jan 9, 2020
nickofthyme pushed a commit to nickofthyme/kibana that referenced this pull request Jan 9, 2020
- Fixes issues involving min intervals for leap years and DST
gmmorris added a commit to gmmorris/kibana that referenced this pull request Jan 9, 2020
* master: (23 commits)
  [Vis: Default editor] Reactify the timelion editor (elastic#52990)
  [Discover] fix histogram min interval (elastic#53979)
  [Telemetry] [Monitoring] Only retry fetching usage once monito… (elastic#54309)
  [docs][APM] Add runtime index config documentation (elastic#53907)
  [SIEM] Detection engine timeline (elastic#53783)
  Filter scripted fields preview field list to source fields (elastic#53826)
  Management - New platform api (elastic#52579)
  Reset region and Account when switching inventory (elastic#54287)
  [SIEM] [Case] Case workflow api schema (elastic#51535)
  Code coverage setup on CI (elastic#49003)
  [ML] DF Analytics Results: adds link to docs (elastic#54189)
  Update schemas boolean, byteSize, and duration to coerce strings (elastic#54177)
  [Metrics UI] Pass relevant shouldAllowEdit capabilities into SettingsPage (elastic#49781)
  [Canvas] Fixes bugs with autoplay and refresh (elastic#53149)
  [ML] DF Analytics Classification: ensure confusion matrix can be fetched (elastic#53629)
  Fix Vega react eslint errors (elastic#54259)
  Remove non existing codeowners (elastic#54274)
  use correct type (elastic#54244)
  [Dashboard] Removing 100% as dshDashboardViewport height (elastic#54263)
  add `examples/` to no-restricted-path config (elastic#54252)
  ...
nickofthyme added a commit that referenced this pull request Jan 9, 2020
- Fixes issues involving min intervals for leap years and DST

Co-authored-by: Marco Vettorello <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Fixes for quality problems that affect the customer experience Feature:Discover Discover Application release_note:fix Team:Visualizations Visualization editors, elastic-charts and infrastructure v7.6.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Discover not showing histogram bars
5 participants