REST API to get the visualization request

[timmolter]

I'd like to be able to query the Kibana web application and get the Elasticsearch request bodies used to build all the visualizations on a given dashboard. See screeny:

Is this possible?

[bruskiza]

Hi,

I did exactly the same.

Copy the content of the request and put in a web request like so:

curl -XGET http://localhost:9200/<index_name>/_search http://localhost:9200/%3Cindex_name%3E/_search -d ‘’

Does this make sense?
It works for me.

#ELK_AWESOME

On 13 Apr 2015, at 2:01 PM, Tim Molter [email protected] wrote:

I'd like to be able to query the Kibana web application and get the Elasticsearch request bodies used to build all the visualizations on a given dashboard. See screeny:

https://cloud.githubusercontent.com/assets/738972/7115146/80d1a8e8-e1e5-11e4-9466-3765e72ef499.png
Is this possible?

—
Reply to this email directly or view it on GitHub #3580.

[timmolter]

@bruskiza Thanks, but that's exactly what I want to avoid: manually copying and pasting. I want to directly query Kibana programmatically (via REST API) to get the ES request body.

[bruskiza]

Locating the visualisation is always a challenge for me.

The Kibana items are actually contained as documents in the kibana index themselves.

http:/localhost:9200/.kibana/_search?pretty http://gosalyn.isignite.co.za:9200/.kibana/_search?pretty

The _id of this index gives clues as to their queries.

and the query field should give you the field you are after.

Not so?

Regards

On 13 Apr 2015, at 4:13 PM, Tim Molter [email protected] wrote:

@bruskiza https://github.com/bruskiza Thanks, but that's exactly what I want to avoid: manually copying and pasting. I want to directly query Kibana programmatically (via REST API) to get the ES request body.

—
Reply to this email directly or view it on GitHub #3580 (comment).

[timmolter]

Given the above visualization screeny, when I find that in http:/localhost:9200/.kibana/_search?pretty, I get:

{"title":"Bro Log Type","visState":"{\"type\":\"pie\",\"params\":{\"shareYAxis\":true,\"addTooltip\":true,\"addLegend\":true,\"isDonut\":false},\"aggs\":[{\"id\":\"1\",\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"type\":\"terms\",\"schema\":\"segment\",\"params\":{\"field\":\"type.raw\",\"size\":12,\"order\":\"desc\",\"orderBy\":\"1\"}}],\"listeners\":{}}","description":"","version":1,"kibanaSavedObjectMeta":{"searchSourceJSON":"{\"index\":\"logstash-*\",\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":[]}"}}
}

, which is not the exact Elasticsearch request body that is provided in the GUI. It's very different than what's given in the Kibana GUI:

{
  "query": {
    "filtered": {
      "query": {
        "query_string": {
          "query": "*",
          "analyze_wildcard": true
        }
      },
      "filter": {
        "bool": {
          "must": [
            {
              "query": {
                "query_string": {
                  "analyze_wildcard": true,
                  "query": "*"
                }
              }
            },
            {
              "range": {
                "@timestamp": {
                  "gte": 1428925280004,
                  "lte": 1428926180005
                }
              }
            }
          ],
          "must_not": []
        }
      }
    }
  },
  "size": 0,
  "aggs": {
    "2": {
      "terms": {
        "field": "type.raw",
        "size": 12,
        "order": {
          "_count": "desc"
        }
      }
    }
  }
}

[rashidkpc]

currently there is no API for this. Adding as a feature request

[nordringrayhide]

+1

[timmolter]

I would like to implement this. Can someone point me in the right direction?

[timmolter]

It looks like in /kibana/src/kibana/components/visualize/spy/_req_resp_stats.html there is an AngularJS reference to a model: {{req.fetchParams.body | json}}. Since AngularJS uses REST API calls, I'm guessing I could also use this REST API call directly. Now I just need to find out what the URL is...

[grifx]

@timmolter Any updates?
I've never used Kibana yet but I project to show some graphs to some customers on my website.
What is the procedure to do this with Kibana?

Regards

[timmolter]

I ended up querying the .kibana ES index directly and parsing the dashboard. It's not very elegant. I hope someone creates a nice API.

Sent from my iPhone

On 01 Jul 2015, at 08:09, Joris Garonian [email protected] wrote:

@timmolter Any update?
I'm never used Kibana yet but I project to show some graphs to some users.
What is the procedure to do this with Kibana?

Regards

—
Reply to this email directly or view it on GitHub.

[gugahoi]

I am also interested in this. Kibana seems like an awesome tool and would be great to have it in the backend and programaticcaly query dashboards/visualizations trough ajax somehow.
@timmolter Any chance you could expand on how you dealt with your problem? Did you just get the data and recreate the graphs in your own page?

[LiorKizer]

@timmolter @luckycse can share how to get the visualisation data?
thanks 👍

[timmolter]

Hi. No problem. I ended up querying the .kibana ES index directly. All the info is there, albeit not in the exact form as shown in the Kibana GUI. I then deserialized the JSON into a hierarchy of Java objects and from there went on my merry way.

So as far as I can tell, forget about querying the Kibana webservice directly via some HTTP REST endpoint. Go directly to the Elasticsearch index and grab the (hidden) .kibana index.

[LiorKizer]

@timmolter, thanks!
I'm also querying the .kibana index inside the ES
did you mange or tried to do it via curl commands? its looks harder to get inside and to extract the vis. the dashboard seems to be an easier story.

[LiorKizer]

as far as I know creating the index itself need to be manually

[JeremyColton]

Hi,
Since you are fetching the graph data as JSON, how are you then rendering the data in a graph? You have found the code used by Kibana and call these javascript libraries yourself?

[sap2000]

Using firebug it was possible to get request used in Kibana 4.1.1 UI. However not all data could be retrived.
Out of many objects only first 10 are in the response.
Below is the request:
curl -X POST --header "Content-Type:application/json" --header "Accept: application/json" -u <user>:<password> 'https://<host>:<port>/elasticsearch/.kibana/dashboard/_search' --data '{"query":{"match_all":{}}}'

Replace /dashboard by /search or /visualization to get respective data. In all cases only first 10 objects were returned. Any Idea how to get next objects or all of them in a go?

[jamesbond111]

Now its the case:I want to create a dashboard exactly the same as kibana on my own,in the dashboard it is only one page and can display some graphs.How can i get the log data in the Elasticsearch,by invoking kibanas API or Elasticsearch Restful API?

[zmistrz]

This is work fine
REQUEST POST http://IP/elasticsearch/_msearch

POST header must be:
kbn-version: 5.3.0

And in POST message:
[INDEX_LIST] [ENTER LINE] [REQUEST FROM SCREEN (timmolter commented on 13 Apr 2015)]
{"index":["logstash-2017.04.28", "logstash-2017.04.29", "logstash-2017.04.26", "logstash-2017.04.27", "logstash-2017.04.30", "logstash-2017.04.02" ........... ], "ignore_unavailable":true, "preference":1496999377554}
{"query":{"bool":{"must":[{"query_string":{"query":"*","analyze_wildca ....

[pmriyazu]

Any updates on this

[ylipentt]

+1

[Piyushpky]

+1

[HarshaLaxman]

+1

[kobelb]

There is an API to do this in more recent versions of Kibana, and we just documented this here: https://www.elastic.co/guide/en/kibana/master/saved-objects-api-get.html

[timroes]

There seem to have been mixed up a couple of requests in this issue already.

Exporting and importing visualizations/dashboards (not the request behind it)

This can be done via the above mentioned API already.

Getting the request a visualization does manually

This can be done via the Inspector for all requests of a visualization or in Discover.

Getting the request a visualization did to gather data via a REST API (the original issue)

We discussed that feature recently, and although it is a legit feature request, we decided, that that kind of Rest API might not be something that makes sense for a broader audience and there are also some technical limitations.

Already today a visualization is not necessarily created from a single Elasticsearch request, but can be created from multiple requests, meaning that REST API would not be able to return a single request body, that you can send, but potentially a series of request bodies. In what way those are combined together to make up the visualization is internal to the Kibana code, and thus just knowing the requests, doesn't necessarily give you the capabilities to from the same response.

That will become even more tricky in the future, since we are currently switching over to use the Canvas expressions for all visualizations in Kibana (see #19813). That way you can have an arbitrary amount of functions in your expression, that will cause requests, and potentially combine them in any flexible way you want. Some of the requests might not even need to be represented in the final visualization at all anymore, but could also just be used to e.g. find out axis extends.

That's why we decided not to build a REST API around that, and closing this issue. This of course doesn't influence anyhow what JavaScript APIs are and will be available to plugin authors, e.g. to gather more insights into the expression being send from a panel action or similar.