Failed to load anchor document

[SSmale]

Kibana version:
6.7.1
Elasticsearch version:
6.7.1
Server OS version:
centOS 7.5
Browser version:
Firefox 69
Browser OS version:
Windows_NT x64 10.0.17134
Original install method (e.g. download page, yum, from source, etc.):
yum
Describe the bug:
Clicking on the view surrounding documents button in the discover page gives the error Error: "Failed to load anchor document.". I cannot find any information on what the anchor document is or how it works. This is only an issue on one of our index types and I cannot see what could be causing the difference.

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[timroes]

Could you please open your browsers Dev Tools (F12) and go to the network tools and post the request/response from the _msearch (or _search) request that's made once you click on the "view surrounding documents" button?

[SSmale]

Attached below

{
  "index": "INDEX_NAME-*",
  "ignore_unavailable": true,
  "NAME_metadata": {},
  "preference": 1573664705100
}
{
  "version": true,
  "size": 1,
  "sort": [
    {
      "@timestamp": {
        "order": "desc",
        "unmapped_type": "boolean"
      }
    },
    {
      "_doc": {
        "order": "asc",
        "unmapped_type": "boolean"
      }
    }
  ],
  "_source": {
    "excludes": []
  },
  "stored_fields": [
    "*"
  ],
  "script_fields": {},
  "docvalue_fields": [
    {
      "field": "@timestamp",
      "format": "date_time"
    },
    {
      "field": "FIELD_NAME",
      "format": "date_time"
    },
    {
      "field": "FIELD_NAME",
      "format": "date_time"
    }
  ],
  "query": {
    "bool": {
      "must": [
        {
          "constant_score": {
            "filter": {
              "ids": {
                "type": "TYPE",
                "values": [
                  "UUID"
                ]
              }
            }
          }
        }
      ],
      "filter": [],
      "should": [],
      "must_not": []
    }
  },
  "timeout": "30000ms"
}

{
  "responses": [
    {
      "took": 2,
      "timed_out": false,
      "_shards": {
        "total": 0,
        "successful": 0,
        "skipped": 0,
        "failed": 0
      },
      "_clusters": {
        "total": 1,
        "successful": 1,
        "skipped": 0
      },
      "hits": {
        "total": 0,
        "max_score": 0,
        "hits": []
      },
      "status": 200
    }
  ]
}

[timroes]

@kertal do you by any chance, have any idea what could be going on here?

[kertal]

@SSmale would it be possible to post the mapping of the index that has problems with context view? and 2 sample datasets (just examples), thx!

GET http://localhost:9200/{nameOfIndex}/_mapping

[SSmale]

https://gist.github.com/SSmale/5507e5cc1014192ccc300ad3b4c3661e

Here is a gist with the mapping and datasets

[kertal]

thx, i will use that data to test

[kertal]

I've tested and it works, so I think this problem is specific to your index pattern and the indices that are queried. Context view first tries to fetch the document you clicked on view surrounding documents. That's the anchor document. For some reason no document is returned in this case. Could you use Kibana's devtools to try out a manual query on the index that doesn't work, like this:

GET /{yourIndex}/_search
{
    "query": {
        "ids" : {
            "type" : "{yourDocType}",
            "values" : ["{idOfTheAnchorDocument}"]
        }
    }
}

thx!

[SSmale]

I have done that and it returns the single record.

[SSmale]

Did you have any other suggestions @kertal?

[kertal]

looking again at your result

"_clusters": {
        "total": 1,
        "successful": 1,
        "skipped": 0
      },
"_shards": {
        "total": 0,
        "successful": 0,
        "skipped": 0,
        "failed": 0
      },

so you're searching 1 cluster, and there was no shard found. I'd be interested in your index pattern, do you use cross cluster search?

[SSmale]

yes we use cross cluster search. Sorry what do you mean by index pattern?

[kertal]

the index pattern defines which ES indices are queried. Since you're using cross cluster search, It seems this search doesn't query all of your clusters? that's why no data in this case is returned. That's why it would be interesting to have an exported version of your index pattern, which is exportable in the Saved Object Management:

https://www.elastic.co/guide/en/kibana/6.8/managing-saved-objects.html

[SSmale]

On my dev machine (the one the data is from) only has a single cluster but we are getting the same issue on our multi cluster too.

I have included one that works and one that doesn't into the gist

[Leaf-Lin]

I bump into this issue in Kibana 7.8.0. I notice the document _id is the culprit.

• If the _id contains only alphanumerical characters, it will give me correct context and showing the surrounding url.
• if the _id contains the special character + or =, it works fine too.
• If the _id contains a slash /, it first converts the _id into html ASCII characters (with %2F) but then resolve into wrong url which subsequently causing "Failed to load anchor document" like this:

To check a surrounding document with id= v/xA69ZXhXRk21sWH56BNnwibvs=

It gives the url like:

https://my_kibana_url/app/kibana#/discover/context/my_index_pattern-*/v%2FxA69ZXhXRk21sWH56BNnwibvs%3D?<extra_filter>

Note the context changed into #xA69ZXhXRk21sWH56BNnwibvs= with missing v/

but it then resolves into:

https://my_kibana_url/app/kibana#/discover/context/my_index_pattern-*/xA69ZXhXRk21sWH56BNnwibvs=?<extra_filter>&type=v

Note the _id change and &type=v at the end of url.

• probably worth to check other special characters.

TL;DR, _id containing foo/bar gives url with context bar, with &type=foo appended in the url.

[ppawel]

I can confirm @Leaf-Lin's finding about the _id. For me a document with _id = TLmBgdeL0GOgAqCX+Q1JSRE6Ijlqg95S71GIGS/JYkc= causes this problem, while others without / don't.

As a workaround we will probably adjust the _id when sending the data to Elasticsearch with fluentd but would be nice if Kibana handled it properly...

For the record, we are on ES/Kibana 7.9.

[kertal]

@Leaf-Lin @ppawel this is a different issue than the one @SSmale reported, so I've exported it to #77443

[kertal]

had another look at it and I'm pretty sure this is an issue with cross cluster search. closing it since in the meantime 7.17 is our last minor of our previous major. feel free to reopen with additional information about your cross cluster setup.

[ppawel]

It was a bug in Kibana's URL handling for anchors, not anything backend side...

[kertal]

There were 2 issue described here, the one @SSmale mentioned, and the one @Leaf-Lin mentioned, which was fixed, and was caused by URL handling