Panel that shows the latest value of a field

[elvarb]

In some cases you want to have a dashboard that shows the latest value of a field. For example you have concurrent connections and want the Ops team to see the current status at a glance. Same if you have calls waiting from the phone system. Would work well with the metric filter also.

Letter size scale to the size of the panel.
Custom font color.
Possibly if it's a number a different color if the value is greater than some value.

[turf00]

+1 This feature would be useful to our team as well. We would like to see the last value of a particular field grouped by another field.

[kjellski]

+1 for this, would be a great addition for our dashboard as well!

It's also usefull for everything that has a state and is logging that state.

[sahilthapar]

👍

[jeremybz]

+1

[webmstr]

I was hoping to work around this in a saved search by setting "size": 0 in the definition, but it's not respected there.

[elvarb]

Grafana has a single stat panel that is exactly what is needed in kibana

[jgogstad]

+1

[andrewvc]

+1

[andrewvc]

this is possible by combining filter, terms, and top_hits aggs, as in the example below. While the agg delivers the document, not the metric (top hits does not allow sub-aggs), that can be extracted in the JS code. See the example query below:

{
  "query": {
    "filtered": {
      "query": {
        "query_string": {
          "query": "*",
          "analyze_wildcard": true
        }
      },
      "filter": {
        "bool": {
          "must": [
            {
              "query": {
                "query_string": {
                  "analyze_wildcard": true,
                  "query": "*"
                }
              }
            }
          ],
          "must_not": []
        }
      }
    }
  },
  "size": 0,
  "aggs": {
    "3": {
      "filter": {
        "term": {"tags": "apache_stats"}
      },
      "aggs": {
        "2": {
          "terms": {
            "field": "@host.raw",
            "size": 5
          },
          "aggs": {
            "1": {
              "top_hits": {
                "sort": [{
                    "@timestamp": {"order": "desc"}
                }],
                "size": 1

              }
            }
          }
        }
      }
    }
  }
}

[rashidkpc]

Once #3546 is in this can be closed.

[yehosef]

I'm curious why is this being approached as an aggregation? It's a simple query. We have a queue that processes log data and has a timestamp value stored that is the latests time processed (we then have a cron that takes the the next chunk of time to process and increases the time reference. For this case, the query would look like:

GET metrics-*/_search
{
  "size": 1, 
  "fields": ["redis.current_time","@timestamp"], 
  "sort": [
    {
      "@timestamp": {
        "order": "desc"
      }
    }
  ]
}

What about an approach like this? It should have a different metric type. Maybe it could be called "single value" and it would have a sort field and direction or querystring field (like a filter). This would let it be more flexible and the interface and query would be very simple.

[andrewvc]

@yehosef the reason being that aggregations can be more easily composed
with other aggregations. You're right that the query is simpler it just
isn't as generic in the context of kibana

On Tuesday, August 25, 2015, yehosef [email protected] wrote:

I'm curious, why is this being approached as an aggregation? It's a simple
query. We have a queue that processes log data and has a timestamp value
stored that is the latests time processed (we then have a cron that takes
the the next chunk of time to process and increases the time reference. For
us the query would look like:

GET metrics-*/_search
{
"size": 1,
"fields": ["redis.current_time","@timestamp"],
"sort": [
{
"@timestamp": {
"order": "desc"
}
}
]
}

What about an approach like this? It should have a different metric type -
maybe "latest".

—
Reply to this email directly or view it on GitHub
#678 (comment).

[yehosef]

@andrewvc - this could be - but this is a very specific situation where I need a single value from the original data, the opposite of aggregations. It's seems heavy/silly to make it be an aggregation just because everywhere else it's an aggregation - when a single value solves this problem better.

[taraspos]

I was able to configure latest response for host with configuration similar as @andrewvc . Check image below.

https://cloud.githubusercontent.com/assets/9948629/9663658/27bedefc-526e-11e5-839f-f64c827a27d2.png

[AnkurMathur14]

It would be really nice if this feature is included in up commimg kibana version

[bvorosadmin]

+1 for the ability to display the latest value.

[richard-hulm]

+1

[steveainuk]

+1

[anoinoz]

+1

[raffis]

+1

[markwalkom]

+1 for this from me, it'd be a great metric addition.

[jeromefroe]

+1

[aalexgabi]

+1

[Bargs]

@karthikbaputj 5.3 #7302

[karthikbaputj]

Thank you @Bargs , Is top_hits only supposed to work on Data table visualization ? If I try to use it on Pie charts it's able to query data but not rendering the chart. Would we be supporting other visualizations in future ?

[Bargs]

@karthikbaputj top hits should work on pie charts. Could you please open a new ticket with details about the issue you're seeing?

[jbgi]

@Bargs it looks like top_hits on non-number fields only works for table and metric (hard-coded!!) :

kibana/src/ui/public/agg_types/metrics/top_hit.js

Line 28 in 84ea502

if (vis.type.name === 'table' || vis.type.name === 'metric') {

Would be nice to allows other visualization plugins to use it...

[Bargs]

@jbgi I agree, it would be nice if each vis could tell top_hits what type of data it supports. Feel free to create a new ticket with more info about your use case and we might be able to figure something out.

[slavik57]

I created a visualization using the demo Kibana app to show the last status for ports:

• I used the table visualization
  • Metric:
    • Aggregation: Top Hit
    • Field: status
    • Aggregate With: Concatenate
    • Size: 1
    • Sort On: @timestamp
    • Order: Descending
  • Split Rows:
    • Aggregation: Terms
    • Field: port
    • Order By: Term

The only thing that I'm missing is to sort the results by the status in case I have a lot of ports and I'm just showing the top 10 results in which case I might miss the errors. But if I show all the ports there is no problem

[Sjaak01]

Came here searching for a way to do this and rather shocked that over 4 years later something basic as this still doesn't appear to be possible without jumping through many many hoops.

[Bargs]

@Sjaak01 what do you need to do that the Top Hit metric does not provide?

[yehosef]

I think the issue is that you have to do some extra steps each time:

• Aggregation: Top Hit
• Field: status
• Aggregate With: Concatenate
• Size: 1
• Sort On: @timestamp
• Order: Descending

When you could just have

• Aggregation: Last Value
• Field: status

For a common use case, it's nice to have "short-hand" or helper widgets so I can use it in the simple way I think about it. I just know that I want the last value, instead of "I want the last value so I have to use one of the aggs to get it, and I have to set some options, so it's the right value, etc."

Additionally, if you then wanted to rewrite it at some point to use a simple query instead of aggregation, it wouldn't affect the user experience - they just want the "Last Value".

[Bargs]

@yehosef after you select Top Hit, the only value you have to specify is "Field". Once you select your field, "aggregate with", "size", "sort on", and "order" all get set to sensible defaults automatically. So I don't see how the current Top Hit metric is any more burdensome than your proposed "Last Value" shorthand. If the issue is with the form simply being too complex for 90% of use cases, we could consider hiding some of the options in the "Advanced" drop down.

[yehosef]

I see what your saying - it seems the current top hits implementation is sort of optimized for the Last Value case. As a note, if you the field you choose is number, the aggregate is not pre-selected to "concatenate" - but all the other types it is (not sure if that's intentional). I don't think I would would hide the options under advanced - I think that confuses the other valid uses of top hits. Personally I think it still would be better to have a short-hand since it seems to be a common use case - it's not intuitive that if I want the latest value I would choose a top hits aggregation. After I know it's there, you're right, it's not that much work to do. But I still think it not so clear. Take a few new Kibana users and ask them to make a metric with the current/latest value of a certain field - I'll be very surprised if they would say "oh, I think I should use the Top Hits aggregation for that." It's clever that it works, but it's not intuitive.

…

On Wed, Jan 24, 2018 at 8:25 PM, Matt Bargar ***@***.***> wrote: @yehosef <https://github.com/yehosef> after you select Top Hit, the only value you have to specify is "Field". Once you select your field, "aggregate with", "size", "sort on", and "order" all get set to sensible defaults automatically. So I don't see how the current Top Hit metric is any more burdensome than your proposed "Last Value" shorthand. If the issue is with the form simply being too complex 90% of use cases, we could consider hiding some of the options in the "Advanced" drop down. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#678 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAJHBR-fd79bHBxYpnXEAdWpgBPDKOFjks5tN3WPgaJpZM4BL_9n> .

[Bargs]

@yehosef I'd recommend creating a new Github issue so the team that works on Visualize can consider it. This sounds like an enhancement request to me, and it won't get much attention buried in this closed issue.

[yehosef]

good point - thanks.

…

On Thu, Jan 25, 2018 at 1:00 AM, Matt Bargar ***@***.***> wrote: @yehosef <https://github.com/yehosef> I'd recommend creating a new Github issue so the team that works on Visualize can consider it. This sounds like an enhancement request to me, and it won't get much attention buried in this closed issue. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#678 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAJHBUbZ96SyYVMau3AV9ECSjclrB7fhks5tN7YIgaJpZM4BL_9n> .

[Sjaak01]

@Bargs It doesn't work on most fields for some reason.

I got various string fields with a text and keyword mapping and for some reason only one of them works (both text and keyword) but some other fields I have, also with a text and keyword mapping, don't display. The fields that don't display work fine in other visualizations. Of course there is data for the time range.

However as somebody else mentioned earlier in some cases you also want to only show the last value, regardless of the time range so even if this would work, it isn't a complete solution.

Not showing anything.

edit: tried with a different index and that seems to be working fine (different data but various field types, all work). Could this be some kind of bug?

[Bargs]

@Sjaak01 I would open a separate issue for your field problems with top hits so the vis team can investigate

[sgpinkus]

@Sjaak01, @Bargs same issue here. Is it missing/null values? Seem a lot like #7189. Can't see how you deal with that from reading above discussion. Any ideas?