Histogram to show current and last week data

[sriramkannan]

I would like to show access log traffic data and would like to compare with last week data in the same histogram. Is it possible? It will be two lines the last week line will be in a shaded color. just like how I have done in graphite in attached image.

[bobrik]

Nope. #645

[rabson85]

What you can do (which is not a nice solution, but one that works for this use case and all others concerning graphing two separate data in one histogram) is to graph two values, in separate documents, using two distinct queries and the Date Histogram.

Specifically you can add the extra (last week) document to ES with the "last week data". You will need some field that identifies "lastweek" docs, vs. "normal" docs (to query for last week data, as opposed to this weeks). Then add the relevant metric, where the Field name is the same in both lastweek doc and this week doc (if it is called Webpage.Pageviews in the first doc, it should be in Webpage.Pageviews in the second).

That way you will have: two docs, where one has the value for this week, and where the other has the value for time - 1 week, and then you can graph these in the standard way.

Make sure to "shift" the datestamp "forward" 1 week on the second document as well. So that that time will correspond to the data you already have.

This will bloat your data, but if you minimize the new docs to only contain the relevant metric, and the timestamp, you are only losing the metadata created for the second doc.

Best of luck, I really hope that #374 comes through soon, with "choose timestamp" to visualize, or choose off-set in time, that would solve this issue as well.

[monotek]

+1

[rcrezende]

+1

[sqpdln]

+1 in Kibana 4. Or maybe this is a duplicate of #2756

[pemontto]

👍

[farina]

+1

[LoZio]

Waiting for this feature!
+1

[baddogdown]

+1

[jomar83]

+1, very important for us

[ameetg]

+1, important feature to have in upcoming Kibana version.

[GrahamHannington]

As a stopgap, until Kibana supports the specifically requested feature (this week's and last week's data in the same histogram), I am using the Date Range aggregation with date math in Kibana 4.1.0 to show this week's and last week's data arranged side-by-side in separate bar chart visualizations in a dashboard, like this:

Last week	This week
Monday	Monday
Tuesday	Tuesday (today)
Wednesday	Wednesday (blank in the example below, because it's in the future)
Thursday	Thursday (as above)
Friday	Friday (as above)
Saturday	Saturday (as above)
Sunday	Sunday (as above)

For example:

(In case you're wondering about the data: it's from a development/sandbox system, not a "production" environment.)

Here's the bucket specification for "Tuesday last week":

The heart of this technique is the date math expression now/w, which rounds back to the start (00:00) of the first day of the current week. Happily - at least, for me, from my cultural point of view - the first day of the week in this context is Monday.

Notes:

• The date math operates on UTC timestamp values, whereas Kibana displays times in my local time (Perth, Western Australia: UTC+08:00). So the date math expression now/w-1w+1d represents Tuesday 8:00am last week, local time.
• Before capturing the example dashboard, I temporarily changed the global Kibana dateFormat setting to ha. Otherwise, the x-axis labels would have been long strings that included the date, rather than, for example, "8am to 9am". I've created issue Per visualization field formatters #4957 for this.
• I wanted to save the dashboard with an appropriate relative date ( from:now-1w%2Fw), but attempting to save a dashboard with a "now"-based date causes a fatal error (issue Saving a dashboard with a now-based date causes a fatal error #4956).
• Each visualization has its own y-axis scale, impeding comparison. I could set a common maximum y-axis value, but have so far chosen not to do so. It's occurred to me to create these visualization definitions programmatically (building the JSON and then indexing the resulting documents in the .kibana index), in which case I could perform Elasticsearch queries to establish an appropriate common maximum y-axis value before creating those definitions. I'm also experimenting with the "split charts" feature to achieve a common y-axis scale.
• Creating the example dashboard involved creating 14 visualizations, which I created via the Kibana UI. If I were to do more of these (say, with different metrics), I'd likely either export/edit/import the JSON for the definitions, or I'd generate the JSON programmatically (possibly based on a "parameterized/placeholdered" template of the exported definitions).
• Regarding that last To value now/w-1w+34h in the buckets screen capture (in particular, the 34h): inexplicably (to me), Kibana rejected the expression now/w-1w+1d+10h. After some cursing and experimentation, I discovered that +34h worked where +1d+10h didn't. Seems like a bug to me.

I'm using variations of this technique to do other comparisons of time intervals. Here's a work-in-progress experiment:

Each (unlabeled 😞 ) bar represents a day of the week. Each chart represents a week (two consecutive weeks). I'm considering kludging a "dummy" bar chart visualization (with no actual data) above this visualization, with days of the week as x-axis labels. If you can get past the lack of labels, this arrangement makes it easy to see differences: for example, more transactions occurred on Wednesday (the third bar) of the second week than on Wednesday of the first week. (Minor point: I've yet to make adjustments in the date ranges of this visualization to account for time zone.)

I'm a relative newcomer to ELK, so, while I hope these details might be useful to others, I'd also appreciate feedback and suggestions; including, for example, "Your technique is fundamentally flawed. Here's why: ...".

[xo4n]

+1

[just5guy]

+1

[TiNico22]

+1
It would be great to compare logs activities between the last days or weeks for example
Thanks to GrahamHannington for the tips

[oryband]

👍 @ofir-petrushka @gadisr @doodyparizada

[samuelmr]

👍

[haiguo]

+1

Additional to comparing two date ranges with the same data precision, it would be nice to compare aggregated data against details: What is my day like compare to average/max of last month's data aggregated to time of day in hours/minutes buckets?

[azrle]

+1

[tatellos]

This is now possible using the timelion plugin! https://www.elastic.co/blog/timelion-timeline

[rashidkpc]

As noted, possible in Timelion

[pittoch]

In date range field, I have the impression that we are limited in the number of parameters .

For example, if I put the following expression , it's ok :
#now/y+1M/M+1d

but if I add parameters , I get an error for:
#now/y+1M/M/w+1d
or
#now-1y/y+1M/M+1d

I really feel that the error is related to the number of parameters , not a syntax error .

Is it possible to exceed this limit ?

I need to recover first Tuesday of every month
Example:
1st Tuesday of the month from 8 PM to 7 AM 1st Wednesday .
Which forces me to accumulate many parameters
I need to
#now/y+1M/M/w+1d+20h
to
#now/y+1M/M/w+2d+7h
but it is refused .

Can you help me ?
Thanks

[tommyorndorff]

+1, would be great to have natively supported in kibana

[gdulus]

+1

[ghost]

+1

[vkroz]

+1

[driperdk]

+1

[SeanSnyders]

+1

[SeanSnyders]

I want to be able to lock certain visualizations on my dash to relative time ranges from 'now', such that I can have a graph for each day for the e.g. past 3 days from 'now' and each of those graphs are plotted on a per-minute time-scale irrespective of the dashboard time range.
This will give me the capability to view current data as well as historical data on the same dashboard.

[vtipparam]

+1

[mindis]

+1

[nani3105]

+1

[heinzmeise]

+1

[dev-shubh]

+1

[guyhaim]

+1

[LTzycLT]

+1

[silva96]

+1

[fjiang423]

+1

[dsager]

+1

[feraudet]

+1

[ot8844]

+1

[AndreAga]

+1

[mimgrund]

+1

[eugenej]

+1

[mohsinkerai]

+1

[pratikvasa]

+1

[fishioon]

+1

[narayanan1993]

+1, much-needed future.

[alexfrancoeur]

Hi All - Time Series Visual Builder offers an offset feature. Creating a series based on the last week of data is as simple as modifying a setting in the metric options. This feature is available in 5.4. I am going to close this issue as we introduced this feature but please let me know if for any reason you'd like to re-open.

(the blue metric is last week here, I forgot to update the label)

[tega90]

Any info on this? +1

[alexanderilyin]

You can export data to Prometheus and use it’s query language to do so.

…

Sent from my iPhone

On Jun 4, 2018, at 05:33, tega90 ***@***.***> wrote: Any info on this? +1 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.